Introduction

As the context for medical education changes and faculty members assume new roles and responsibilities[1], faculty development becomes essential for the success of both individual faculty members and the academic institution as a whole[2,3]. In this changing context, universities require committed, competent, and effective faculty members who can respond to multiple expectations, engage in knowledge-producing activities, and maintain the highest level of quality in their work[4].

Universities and academic centers feel pressure to address and balance multiple missions: knowledge production through research, knowledge dissemination through teaching, and knowledge application through service. Faculty members play a crucial role in addressing the missions of academic centers[5]. As the quality of universities is related most closely to the performance of faculty members, support of the latter is paramount. Thus, faculty development is a key strategic lever for ensuring institutional quality and supporting institutional change[6].

Faculty development was defined earlier as “the holistic development of the faculty members”. Since the missions of academic institutions and the roles of faculty members expanded, the definition of faculty development has expanded as well to include a much broader range of concerns. Faculty development implies those activities that are undertaken by academic staff to encourage individual and institutional growth[6-8]. The broader definition of faculty development is: ‘‘Any planned activity to improve an individual’s knowledge and skills in areas considered essential to the performance of a faculty member  in an academic institution (e.g. teaching skills, administrative skills, research skills, clinical skills)’’[9].

In recent years, academic institutions have introduced and implemented innovative aspects into the medical education. Several examples are: the adaption of active learning and student-centered strategies, e.g., problem-based learning, self-directed learning, and outcome-based education; change in clinical teaching, e.g., shift from bedside to ambulatory and community setting; and awareness of patient safety, medical ethics and community development. These curricular changes require appropriate and relevant academic development initiatives to meet the needs of stakeholders – academic institution, leaders, faculty members, and students. In response, faculty development has been suggested as an effective approach in order to facilitate curricular innovations. Therefore, there is a need to provide a comprehensive definition of faculty development in order to track its important role in curriculum redesign[10].

Different approaches may be used to assess faculty development programs at a given academic center. Faculty development may have different orientations to provide individual, instructional, professional, or organizational development[6,8,11-13]. One comprehensive approach may be seen in the broad definition of faculty development by Michelle M.L. et al [9]that is: “the personal and professional development of teachers, clinicians, researchers and administrators to meet the goals, vision and mission of the institution in terms of its social and moral responsibility to the communities it serves”. Given the broad aspects of this definition, we apply a comprehensive pedagogical and managerial approach in order to facilitate assessment of faculty development activities within institutions with hierarchical structure.

With that introduction concerning the concept of faculty development and its current approach in mind, we will then look at research addressing the progression of faculty development. Finally, by applying that comprehensive approach we will describe the methodological and psychometric aspects of developing the instrument.

Why the need to develop a new instrument

Having discussed faculty development and the needs for it, we now turn to the designing process for faculty development programs to meet the curricular needs. The context of each academic institution is unique, therefore, there is no wide-range or “quick fix” model for faculty development[9]. The unique institutional settings in which faculty development occurs affect the way and criteria for assessing the quality and effectiveness of programs. Thus, devising a structure may facilitate the process of planning and assessing faculty development activities. This may be possible by putting generalizable principles of faculty development together[8,14]. This would not only provide the possibility to guide, design and implement a program, but it can also help the faculty developers, administrators, and leaders as the main stakeholders to assess and constructively critique the implemented programs.

Based on a literature review, we could not find a tool that provides the means to assess and analyze the whole system of faculty development. Therefore, we deemed it necessary to develop a new instrument. We used the established and generalizable principles of faculty development, applied the findings of qualitative content analysis of interviews, and reviewed internal documents to develop a comprehensive instrument. So, our instrument is a synthesis of selected works on faculty development, besides our own efforts.

Systematic approach to faculty development in an integrated context for Iranian medical universities

The reform involving the integration of medical education into the Iranian healthcare system was implemented by the Ministry of Health and Medical Education[15]. One of the main reasons for the integration of medical education and the health care system in Iran was to improve the quality of health personnel training in general and medical education in particular, by altering it from a traditional form to a more community oriented education[15,16]. The reform was also aimed to challenge the quality of medical education. The reform brought much opportunity as well as some challenges to medical schools. In term of faculty development activities it led to the formation of Educational Development Centers (EDC). These centers were established in order to plan and implement faculty development initiatives and with the intention to prepare faculty members for new and diverse roles and responsibilities. EDCs and medical universities were asked to design and implement well-organized and effective faculty development programs. Although there are quite a lot of faculty development activities being conducted in Iran, very little assessments were conducted. Without program assessment, it is difficult for decision-makers to support programs and allocate resources to these purposes.

The purpose of this study is to report on the research activities that led to the development of an inclusive assessment tool for faculty development in Iranian medical schools. The instrument covers several components of faculty development program, from the planning and implementing to the evaluation process. This broad approach examines different aspects: individual, professional, instructional, and organizational. This approach differs from the vast majority of faculty development programs which focus mainly on one aspect, e.g., instructional development of faculty.  Therefore, our aim is to embrace faculty development in its broadest context, including teaching, research, administration, leadership and self development.

In this paper we report the development and validation of a national instrument that provides faculty developer/ medical educators with a tool to measure the state of their school’s faculty development activities. It also provides useful information about a systematic approach to faculty development activities for proposing, developing, and implementing an effective faculty development program.

Methods

We conducted a three-phase study. In the first phase, to identify the concepts and define the content of the instrument we designed a search strategy through which involved a review of the literature on several of the library’s online databases and cross reference check. Our bibliographic search strategy was based on various keywords including “faculty development”, “staff development”, “medical teacher training”, “and comprehensive approach”. This phase was followed by a series of interviews with experts. In the second phase, we conducted a pilot test of the preliminary instrument in order to validate and prioritize the generated items into a manageable instrument. After that, in the third phase, the research group summarized the main themes and then identified the main categories to be addressed in the instrument.

First phase: Definition and Development of the instrument content

Deciding whether a self-developed instrument is valid is done by assessing its content validity. Content validity checks how adequately the items of an instrument match the concepts being measured. This often depends upon the opinion of experts who decide whether the test adequately represents the domain of interest[17]. We started developing the content of the instrument with an informal group discussion. Two meetings were hold to discuss the findings from retrieved literature and documents. While developing an outline of the instrument, we also developed the interview guide which we used subsequently to interview expert key informants.

The key informants were: vice chancellors for research and education, deans and vice deans of medical schools, EDC directors, department chairs, educational experts, and teacher training staff. All interviews were tape recorded. Using qualitative content analysis interviews then were transcribed and analyzed in a process whereby multiple reviews went back and forth within the text; at last the themes (indicators) were developed. The final step taken in the analysis was developing a conceptual framework. The conceptual framework which mainly emerged from interviews findings and review of literature gave opportunity to group together similar items. From all of these activities an initial item pool of 150 statements was generated.

During the process of content development and content-related validity, a group (five persons) of EDC faculty members with expertise in the area of faculty development, including head of medical education department, director of EDC, junior and senior lecturers, met to agree on an initial list of statements for inclusion in the instrument. They determined areas where there was consensus, as well as differences of opinions. They reduced the 150 statements to 115 by eliminating repetitive materials and consolidating some. These were individually reviewed by colleagues in another EDC, and based on their feedback the instrument was further reduced to 98 statements. This kind of group discussion provided credibility to the process of instrument development.

Content validity refers to the appropriateness of content and composition. If an instrument has Content validity it means that the instrument seems to be measuring what it intended to. This was the step above performed by the expert group discussions. We also performed a pre-pilot test with some local faculty members in order to confirm Content validity from the perspective of faculty members as survey-takers.

To summarize the first stage of instrument development, first we conducted a thorough literature review and semi-structure interviews in order to identify appropriate concepts and framework. We then combined the findings with data from expert informants. Other experts then verified and changed the statements. The content validity of the instrument was tested by these multi-step processes. Finally, this several iterations process led to the second stage of development.

Second phase: Pilot test of the preliminary instrument

To validate and prioritize the 98 questions into a more manageable instrument, we conducted a pilot study. We examined whether the items should have been written in the form of questions. We also examined the appropriate response options with their respective scale. We decided that since the instrument dealt with opinion and experiences of respondents about the existing situation of faculty development programs, the possible response options should have included questions with a continuum scale.

Copies of the 98-question preliminary instrument were mailed to eighty five faculty members who were then asked to return them directly to their respective EDC. The pilot samples were chosen from eight medical schools of three different sizes. Based upon the review of relevant factors that appeared to consider for faculty development programs, a series of questions were compiled. The respondents were asked to rate the question on a three-point scale (‘agree’, ‘no idea’, ‘disagree’ respectively corresponding to score 3, 2, and 1) according to the relevance of each question within faculty development programs.

Third phase: Developing categories and indicators

A group of three researchers then summarized the main themes and identified 6 categories: Input, Output, Process, Evaluation, Faculty roles, and Obstacles and challenges. The first four categories contained 16 indicators with 69 questions, while the other two categories contained 16 questions (Table 1). Finally, the instrument comprising 85 questions, contributed in the assessment of designing, implementing and measuring faculty development activities.

Table 1: Factorial Analysis of the Instrument

Results

Qualitative data obtained from interviews, documents, and literature were coded, categorized, and tabulated using content thematic analysis by the main author and also a third-party investigator, to ensure its trustworthiness. To report the qualitative findings we did not use direct quotes from the interviewees because no single quote can fully illustrate a category. Six categories emerged from the qualitative analysis with 16 indicators and a total of 85 questions including:

• Input category with six indicators: performing needs assessment, partnership in needs assessment, using diverse sources of information, addressing diverse needs, appropriateness of resources, and diversity of contents and topics
• Process category with five indicators: diversity of implementation methods, access to programs, using active teaching, and learning strategies, continuity of programs, and incentives
• Output category with three indicators: obtained skills and abilities, satisfaction, and achievement of objectives
• Evaluation category with two indicators: program evaluation, feedback system
• Faculty roles addressed in programs category with six indicators: teaching, research, practice (clinical/allied health), administrative roles, personal development, and professional community services
• Obstacles and challenges category with nine indicators: inadequacy of managerial and/or organizational support, inadequacy of department and school support, inadequacy of faculty partnership in decision making, fragmented and parallel structures dealing with faculty development activities, instability of management in the decision making process
• Resistance to change, lack of follow-up activities, lack of standards for faculty training, and lack of systematic approach to the faculty training process

From all of these activities an initial item pool was generated with 150 statements.

Of the 150 items generated from a review of the literature, interviews of key informants with senior faculty leaders who have extensive knowledge, and experience performing faculty development activities and group discussion with faculty members, only 98 items in the item reduction phase were chosen. In the pilot testing responses were received from EDCs. Fifty responses were considered for analysis. On a mean score of 1-3, 45 questions were rated ≥2.5, 29 were rated ≥2.00 but ≤2.49, and the remaining 21 were rated ≤2.00. Several of the highest ranked questions came from the end of the instrument, suggesting that the length of the questionnaire was not a problem. The reliability was high with an alpha of 0.83. Based on the results of this analysis 85 items with a high mean score (from 98 items) were considered relevant and were retained for the final version of the instrument.

Exploratory factor analysis was done. The Kaiser criterion (Eigen values>1) suggested six factors Table 1. Furthermore, to find out if there were item with low correlation we used item analysis. The results of item analysis could not lead to removal or changes of items in the instrument. Cronbach’s alpha coefficient of the instrument was .90. Twenty two items had high correlation (>0.9) in the scale. Deletion of items with low correlation did not significantly improve the internal consistency. From an item analysis characteristics viewpoint, there were not item difficulty (number of very difficult items) and item discrimination (number of poorly discriminating items). Therefore, we did not eliminate any items on the final instrument.

In summary, the three-phase process of instrument development was conducted by applying a qualitative content analysis and involving stakeholders from the relevant area of expertise in Iranian medical universities. These enquiries generated 85-question instrument with 16 indicators and 6 categories with validity for medical universities in Iran. The instrument had a reliability Cronbach’s alpha coefficient of 0.90 in that administration. The final instrument contained two parts: part one, which covered the socio-demographics and academic details of respondents, and part two which addressed a short explanation of some indicators and then items in the form of short statements.

We conducted a survey among faculty members with top leadership positions. Copies of the 85-question instrument were mailed to 280 faculty members. The question responses were given on a three-point scale as ‘yes’, ‘to some extent’, and ‘no’ or as ‘fulfilled’, ‘partially fulfilled’, and ‘not fulfilled’ (scored 3, 2, and 1 respectively). These questions were intended to determine the presence or fulfillment of specific factors considered relevant to faculty development programs; for example, answering ‘yes’ meant that the indicator existed or was addressed within faculty development programs.

Discussion

Assessing conceptual and experiential subjects such as faculty development program components is complex and requires appropriate conceptual and theoretical frameworks with which to guide the assessment process. The validity of the instrument based on established and generalizable principles of faculty development was valuable in guiding the development process. In addition, that process was further followed by group discussions with experts. In the development process, although it is time consuming, it is imperative to include as wide a range of views as possible (faculty members, stakeholders and providers of programs) in ensuring that items tap relevant aspects of the characteristics they are intended to measure. This approach also promotes the content validity of the scale.

The results reported in this study provide support for internal consistency of the instrument. In addition, content development and validity of the instrument support the instrument’s items in terms of their objectivity for assessing faculty development activities from a broad perspective—managerial and pedagogical.

On the other hand, the scope of the faculty development instrument is to cover faculty development activities in its broadest context including teaching, research, administration, leadership, and self-development. We hope that this instrument provides stakeholders with a new way to approach faculty development activities. The instrument can guide in building and detecting effective faculty development programs and in mapping the existing situation. It can also be helpful in establishing and evaluating successful faculty development programs.

However, this study describes the development of an instrument used for assessing faculty development activities within Iranian medical education settings. We look forward to those researchers in the field of medical education and faculty development to test it in other contexts in order to further develop it.

Acknowledgements

The authors are grateful for financial and administrative support from the directors and staff of the Medical Education Research Center at Isfahan University of Medical Sciences and the National Public Health Management Center at Tabriz University of Medical Sciences. Many thanks go to the faculty members who participated in interviews and on group discussions, and those who completed and returned the questionnaire.

References

1. Harden RM, Crosby J: AMEE Guide No 20: The good teacher is more than a lecturer – the twelve roles of the teacher. Medical Teacher 2000, 22: 334-347.
2. Steinert Y: Staff development. In A practical guide for medical teachers. Edited by Harden MR, Dent AJ. Edinburgh: Churchill Livingstone; 2005:390-399.
3. Svinicki M: Faculty development: an investment for the future. In Field guide to academic leadership. Edited by Diamond RM, Adam B. San Francisco: Jossey-Bass; 2002.
4. Skeff KM, Stratos GA, Mount JFS: Faculty development in medicine: a field in evolution. Teaching and Teacher Education 2007, 23: 280-285.
5. Sorcinelli MD, Austin AE, Eddy PL, Beach AL: Creating the future of faculty development: learning from the past understanding the present. Bolton, MA: Anker; 2006.
6. Carole JB, Lisa W, Wendy V, William J: Evaluating faculty performance: a systematically designed and assessed approach. Acad Med 2002, 77: 15-30.
7. Steinert Y, Mann K, Centeno A, Dolmans D, Spencer J, Gelula M et al.: A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Medical Teacher 2006, 28: 497-526.
8. Steinert Y: Faculty development in the new millennium: key challenges and future directions. Medical Teacher 2000, 22: 44-50.
9. McLean Michelle, Cilliers Francois, van Wyk Jacqueline M: Faculty development: Yesterday, today and tomorrow. Medical Teacher 2008, 555-565.
10. Jolly BC: Faculty Development for Curricular Implementation. In International Handbook of Research in Medical Education. Edited by Cees van der Vleuten, David I Newble. DORDRECHT/ BOSTON/ LONDON: Kluwer Academic Publishers; 2002:945-967.
11. Bardley AB, Peter JC, Walter LF, Lawrence JH, Duane DM, Robert BP et al.: A comprehensive approach to faculty development. American Journal of Pharmaceutical Education 2006, 70: 27.
12. Wilkerson L, Irby DM: Strategies for improving teaching practices: a comprehensive approach to faculty development. Acad Med 1998, 73: 387-396.
13. Steinert Y, McLeod P M, Boillat M, Meterissian S, Elizov M, Macdonald ME: Faculty development: a ‘Field of Dream’? Medical Education 2009, 42-49.
14. Steinert Y, Mann Karen V: Faculty Development: Principles and Practices. Journal Vet Med Educ 2006, 33: 317-324.
15. Azizi F: The reform of medical education in Iran. Medical Education 1997, 31: 159-162.
16. Marandi A: Integrating medical education and health services: the Iranian experience. Medical Education 1996, 30: 4-8.
17. Shea JA, Fortna GS: Psychometrics Methods. In International Handbook of Research in Medical Education. Edited by Geoff Norman, Cees van der Vleuten, David I Newble. DORDRECHT/ BOSTON/ LONDON: Kluwer Academic Publishers; 2002:97-126.