Aspect: Achievement of qualitative targets for 'knowledge and understanding'

The programme ensures through its design, teaching/learning activities and examination, that doctoral students who have been awarded their degrees show broad knowledge and understanding both within their third-cycle subject area and for scientific methodology in the third-cycle subject area.

The scheme presented in the self-evaluation documents that courses and training in the subject area and research methods run parallel to thesis work. Research and writing of the thesis, compulsory courses, and elective courses (i.e., workshops, seminars, and conference participation) are an integral part of the doctoral trajectory. Systematic reviews of the thesis progress are conducted at the midterm presentation (50%), at two presentations of the doctoral work made at the Higher Seminars – one after one year (25-30%) and the second after three years (70-80%) – and finally at the defence (100%). At the final defence, international committee members are usually present.

The activities that guarantee that broad knowledge and understanding is achieved and progress is made during the doctoral trajectory include the individual study plan, methodological seminars, national and international conference participation, and individual supervision. Moreover, the higher education institution encourages doctoral projects that explore design and experimental writing as a mode of research and knowledge production. The assessment panel sees these activities and structure as fit to guide students and to achieve the targets of knowledge and understanding in a satisfactory way. Further attention to the mutual relationship between these elements of supervision could enhance the programme even more.

The panel notes that doctoral students of the programme take around one-and-a-half years more net study time to complete their degree than doctoral students in architecture in other higher education institutions in Sweden. In the interviews, this was explained in part by an earlier supervisory culture and this is expected to improve with recent changes in supervisory staff and strengthened review structures. One reason given for the longer completion rates is that there seems to exist a certain tension between the kind of experimental research done and the expected length of doctoral trajectories. Another reason has to do with the ambitions and high standards of the higher education institution. Moreover, doctoral students who combine professional work, for example at a municipality, and thesis work obviously have a longer gross period of study. The assessment panel advises the higher education institution to develop a policy that makes the ambitions explicit to doctoral applicants including the potential of this kind of work and study trajectory, so it will in future be clear what aims are set and whether they are met.

The programme's design and teaching/learning activities are systematically followed up to ensure achievement of qualitative targets. The results of the follow-up are translated, when necessary, in actions for quality improvement, and feedback is given to relevant stakeholders.

The main instruments for systematic follow-up are the individual study plans and the study plan meetings, Higher Seminars, supervisorial feedback, and departmental ’research training meetings’. In the follow-up of the progress of doctoral students, the digital individual study plan is obligatory and considered to play an important role. In cases where progress is lagging, the supervisor and doctoral student are invited to clarify the reasons. In addition to presentations at Higher Seminars, the programme director calls for ’research training meetings’ once or twice per term where the audience of programme director, supervisors, and doctoral students evaluate the students’ progress and provide feedback. Moreover, there are individual talks between the programme director and the doctoral students. The panel notes that direct feedback to the doctoral student is guaranteed by the current system in a commendable way.

When it comes to systematic follow-up of the ResArc activities, doctoral students give feedback on the ResArc courses. Higher seminars and ResArc courses are complimentary. The interviews indicated that the structure and content of the third-cycle programme are not generally well understood by doctoral students, which suggests that the programme syllabus could be more effectively disseminated and utilised.

While a valuable structure for evaluation and feedback is provided by the higher education institution, the assessment panel believes that more attention to the structural role of the individual study plan could enhance the programme even more. The presented examples of study plans differ. Most of them are quite clear regarding the courses planned and completed, progress and thesis abstract, but not very explicit in the methodological approach to the subject matter (i.e., the plans do not clearly elaborate research methods). During the interviews, the assessment panel also received consistent answers about the need for further refinement with the shift to electronic individual study plans.

Regarding the systematic follow-up of training of research methods and a methodological approach to thesis work, the staff informed the assessment panel in the interviews that the development of scientific methodologies is supported by ResArc courses, supervisor-student discussions (e.g. advice to take certain courses etc.), actual step-by-step writing exercises, and the supervisors' help in managing each student's process. There is an understanding that progress is a collective responsibility. This 'co-supervising' is highly valued by the assessment panel and can enhance regular feedback from other relevant stakeholders and as such deserves structural attention in the future.

Aspect: Achievement of qualitative targets for 'competence and skills'

Through its design, teaching/learning activities and examination, the programme ensures that doctoral students whose degrees have been awarded can plan and use appropriate methods to conduct research and other qualified tasks within predetermined time frames, and in both the national and the international context, in speech and in writing authoritatively, can present and discuss research and research findings in dialogue with the academic community and society in general. Doctoral students are able to contribute to the development of society and support the learning of others within both research and education and in other qualified professional contexts.

The main instruments to guarantee that the doctoral students achieve good planning skills and can choose appropriate methods are discussions with the supervisors, participation in (higher) seminars, presentations during ‘research training meetings’, and research days. Seminars support students in the choice of appropriate methods. The interviews made it apparent that all supervisors have access to doctoral students' progress seminars (30%, 50%, etc.) and that this is a collective process.

In addition, doctoral students prepare publications, contribute to conferences, participate in conference preparations and develop paper reviews that train the students in oral presentations, reading, and writing within a set time frame. The higher education institution also promotes international exposure by encouraging doctoral students to publish and take part in international conferences. In addition, lecturers from abroad are invited. Each student receives 10,000 SEK/year to attend national and international conferences, both nationally and internationally. The assessment panel hopes that this support will remain after ResArc funding ends. Overall, the interviews confirmed that the doctoral students have good connections with international networks, although it was noted that it would be helpful to have a senior researcher specifically dedicated to facilitating this for the doctoral students.

During their research trajectory, the doctoral students also gain experience by teaching first-cycle and second-cycle students, which evolves their competence and skills in supporting the learning of others.

Doctoral students are further prepared to contribute to the development of society in general by the he higher education institution's continuous social engagement - for example, through two large research and collaboration platforms that integrate research with society and practice.

The programme is followed up systematically to ensure that the design and teaching/learning activities are high quality and that the doctoral students achieve the qualitative targets. The results of the follow-up are translated, when necessary, into actions for quality improvement, and feedback is given to relevant stakeholders.

The process described above guarantees systematic follow-up and feedback to the doctoral students. The individual study plan helps frame time schedules and deadlines. The programme director plays a crucial role in making sure that progress of the individual doctoral student is actually followed, and when encountering obstacles, doctoral students are helped and receive feedback regarding competences and skills.

Aspect: Achievement of qualitative targets for 'judgement and approach'

Through its design, teaching/learning activities and examination, the programme ensures that doctoral students who have been awarded degrees show intellectual independence, scientific probity and the ability to make research ethics assessments. The doctoral student also has a broader understanding of the science's capabilities and limitations, its role in society and human responsibility for how it is used.

According to the self-evaluation, the qualitative targets of intellectual independence, the ability to demonstrate scientific probity, disciplinary rectitude, and the ability to assess research ethics are ensured and promoted throughout the doctoral trajectory, and encouraged and evaluated in meetings with the supervisors and peers, seminars, and actual work on the thesis. Ethics as well as the capabilities and limitations of scientific research are explicitly discussed in a series of mandatory Higher Seminars. Seminars and architecture-specific courses on concepts and theories, as well as communication of knowledge within architectural research, are considered important for developing an independent position and the capacity for scientific probity and ethical responsibility.

The programme is followed up systematically to ensure that the design and teaching/learning activities are high quality and that the doctoral students achieve the qualitative targets. The results of the follow-up are translated, when necessary, into actions for quality improvement, and feedback is given to relevant stakeholders.

A whole range of activities is offered in which doctoral students are engaged. The mid-term review, final seminar, and final dissertation defence, in which external reviewers take part, ensures the overall quality of the programme. To maintain such a broad range of topics and approaches, a critical mass of doctoral students is needed. Therefore, the panel sees the support of activities provided through funding – e.g. the ResArc seminars and new doctoral students – as crucial.

To make sure and follow-up that doctoral students achieve the qualitative targets, a recent change was made to align local goals with national goals (i.e., the qualitative targets that doctoral students should meet either through their thesis work or through compulsory as well as elective courses and seminars).

The interviews indicated that collaboration with stakeholders and follow-up of qualitative targets with respect to relevance to society are things that will be developed in the anticipated revision of the programme syllabus in 2017–18. In 2016–17 the syllabus was refined with a view to more clearly specifying 'judgement and approach' in relation to the subject area. The interviews clarified that stakeholders are understood generally to encompass the building sector, the public, expert groups such as the Swedish Architects Association and political organisations, as well as funding bodies such as Formas, which supports initiatives such as the ResArc initiative. This expansive definition is commended by the panel.

Overall assessment of the aspect area 'design, teaching/learning and outcomes'

In the overall assessment, the aspect area 'design, teaching/learning and outcomes' is deemed to be satisfactory.

The higher education institution offers an effective structure ensuring that doctoral students achieve the qualitative targets of knowledge and understanding, competence and skills, as well as judgement and approach through individual study plans, methodological seminars, national and international conference participation, and individual supervision. Regarding the training of research methods and a methodological approach to thesis work, the assessment panel values that the development of scientific methodologies is supported by ResArc courses, supervisor-student discussions (that include advice on which courses to take), actual step-by-step writing exercises, and cross-referral by supervisors. This 'co-supervising' is received very positively and as exemplary. An international orientation and performance adds to this positive framework. Regarding the aspect 'knowledge and understanding', the assessment panel sees potential areas of improvement regarding the use of the individual study plan, particularly with regard to methodology often not being dealt with in the individual study plan.

The higher education institution stimulates doctoral projects that explore design and experimental writing as a way of research and knowledge production. Further attention paid to the mutual relationship between these elements of supervision could enhance the programme even more.

Based on the interviews, the panel has some concerns that the higher education institution's attitude toward the average length of completion of its doctoral students' degrees is contradictory. On one hand, it is explained as a relic of an earlier supervisory attitude, which has been changed; on the other hand, it is described as evidence of ambition and rigour. The assessment panel encourages the higher education institution to reflect on this and establish a clear position with regard to it.

In summary, the panel commends the robust structures for supporting doctoral student progress, an expansive understanding of relevant stakeholders, and good evidence of interaction and sharing of experience between supervisors.