Development of a Methodology for the Design and Management of Academic Strategy – A Holist Approach using Multi-criteria Analysis Techniques

DOI: http://dx.doi.org/10.24018/ejers.2020.0.CIE.2303 December 2020 1 Abstract—Aim of the current research paper is to propose an innovative solution for the problematic of the holistic management of an academic strategy. The systematic bibliographic surveys conducted showed that the combination of BSC method together with a multitude of MCDA techniques constitute the most important tools for this issue. Thus, we propose a holistic process-based methodology for the management of an academic strategy which spans from its design and oversight, to interpretation issues of the academic classification of departments of Universities or between Universities where assembly bodies (Quality Assurance Unit, HAHE) are active. We claim that our methodology is of particular importance and that its use will highlight the operational quality of well organised Universities.


I. INTRODUCTION
The overall assessment of the design and management of an academic strategy as well as the interpretation of the classification of academic performance, constitutes admittedly a complex issue as a multitude of factors will need to be considered. Namely these are strategic decisions that are being taken at the highest level of academic administration. Each Higher Education Institution (HEI) fulfills to a different extent the criteria set in Greece by the Hellenic Authority for Higher Education (HAHE) however no Institution meets adequately the entirety of the criteria. The fulfillment of the qualitative criteria can either be subject to quantitative numbers that are defined by structured processes or in numbers whose assessment is subject to the experience and opinion of the decision makers. The latter are to a certain degree semi structured or even unstructured processes. The problem of management of academic strategy is therefore a multidimensional, complex decision problem whose solution depends on both qualitative and quantitative data and which, like any human decision, is characterized by considerable subjectivity. It therefore follows that in cases where the decisions for the academic strategy are either not adequately documented or not documented at all, it is deemed necessary to dissect the value system of the decision maker and to verify his/her decision against the criteria he/she invoked.
According to the literature review we performed, BSC alone or combined with other strategic management techniques as well as various types of multiple-criteria decision analysis (MCDA) are methodologically successful tools for the design and management of an academic strategy. To this day, how this can be actually applied in practice has only partially been highlighted in international bibliography relevant to the solution of some aspects of the plexus "Design, Implementation & Monitoring of the academic strategy of a University". The aim of our paper is to synthesize the preexisting knowledge on the subject into a holistic methodology and to add to it the aspect revealed to us by the values system of those determining the academic performance of a University's department or comparing those of many Universities. Especially when these are not satisfactorily documented and therefore call for interpretation and verification. Such endeavor has yet to be published in the international bibliography which is indicative of the contributing potential of this paper.
In our effort to determine an optimal solution for the academic classification of the departments of a single University or a multitude of Universities we used the UTASTAR method of the Preference Disaggregation Approach (PDA). The latter assumes that the decision maker reaches to his/her decision based (consciously or unconsciously) on a certain values system and preferences. It analyses the relationship between the decisions and the performance of the alternatives against the criteria and thus it detects the way through which these decisions are taken by developing a criteria synthesizing template. Essentially, the difference from other multiple-criteria approaches (multi-criteria value, dominance relations) lies on the fact that the latter synthesizes the data of a given problem in order to reach to a final result, while PDA analyses data in order to identify the template which best reflects the values system and preferences of the decision maker. In order to accurately determine the template in question the collection of information relevant to the values system and preferences is required. Additionally, the collection and analysis of a sufficient data set of examples of decisions taken by the decision maker needs to be performed. This information usually consists of the actual decisions without any further parameter as to how these were taken and are manifested in various forms such as a monotony scale (classification and taxonomy of alternatives) or through an indicator (the amount of times that an alternative is preferred over others).
The examples can be previous decisions taken, a small but representative set of imagined alternatives or a small but representative subset of the alternatives under consideration which are clearly outlined by the decision maker.
For the purpose of this paper we shall briefly refer to relevant bibliographical data, we will present the steps of the methodology which we propose for a holistic management of an academic strategy, we will make mention to its previous partial application in relevant papers of ours and finally reach to our conclusions.

II. LITERATURE REVIEW
The international bibliography is rich with cases where the BSC & MCDA techniques are applied in numerous sectors of economic and societal areas of activity of organisations, businesses and Universities. For the purpose of this paper however, we will only focus on the most prominent bibliography relevant to the management of academic strategy.
The paper of Fahmi Fadhl et al. [1] examines studies in renowned journals focusing on the Balanced Scorecard in Higher Education Institutions technique. This review showcases the potential application of BSC in Universities. In her paper [2] M. Hladchenko focused on the comparative analysis of the Balanced Scorecards of four higher education institutions and aimed at determining a general BSC framework for the latter.
The authors V. Umashankar & K. Dutta focus their study [3] on the Balanced Scorecard (BSC) and present the way through which it should be applied on higher education institutions in India. Their paper is based on existing bibliography relevant to the notion of Balanced Scorecard as well as on its application in higher education. The paper by M. Peris-Ortiz et al. [4] presents the adoption of performance measuring systems by some Latin American Universities as tools for strategic oversight. As such the Balanced Scorecard (BSC) was selected. The paper by C. Papenhausen & W. Einstein [5] showcases the increasing tendency of United Kingdom Universities to implementing performance management. In his article [6], D.F. Beard presents a series of successful implementations of BSC in Universities throughout the world. From the bibliography as well as the study of specific success stories it becomes evident that BSC can be used by higher education institutions not only for quality oversight purposes but also for the improvement of the Universities management. In the paper of F.F. Al-Hosaini & S. Sofian [7], which is of particular importance to us, a concise bibliographical analysis relevant to the application of BSC in Universities in its four contextual dimensions is presented.
There are indeed numerous research papers focusing on the combined applications of the MCDM and BSC methods in various areas, but very few of them focus on the evaluation of Universities. Hashemkhani, Zolfani και Radfar (2011) presented a review article concerning the selection of the best hybrid models of the MCDM and BSC methods [8].
The results indicate that ANP and VIKOR are superior to AHP and TOPSIS combined with BSC, while DEMATEL is suitable for the calculation of the cause and effect relations between the BSC perspectives. The bibliography review also revealed that in many researches the MCDM methods used (Fuzzy) AHP, (Fuzzy) ANP for the calculation of the indicator's weight (Dytczak & Ginda 2009 [9], García, Melón et al. 2010 [10], Azimi et al. 2011 [11], Timoshenko 2008 [12]) while in other researches conducted, the DEMATEL method was used based on the cause and effect relationship between the perspectives and the indicators.
In the study of H. Zolfani, and S. Ghadikolaei, A. [13] three MCDM methods were applied in combination for the assessment of private Universities. Initially DEMATEL was used to assess the cause and effect relations between BSC perspectives, the next step consisted of the application of ANP for the identification of the important criteria and their weight while in the end VIKOR was applied so as to compare selected Universities as study cases and for their classification.
Finally, in our recent paper [14] the combined application of the BSC and Utastar techniques was attempted for the first time. This in an effort to interpret the decision makers' reasoning as well as the revelation of their values system regarding the classification that they conducted in seven Greek Universities according to selected criteria of their Internal Quality Assurance Systems.

III. PROPOSED HOLISTIC METHODOLOGY FOR THE DESIGN AND MANAGEMENT OF ACADEMIC STRATEGY
Prior to outlining our methodology, we shall make mention to some of its core aspects.
In order for the BSc to be applied in the higher education area [15] the most useful measurements corresponding to its four dimensions need to be defined. Obviously, in order to set out an exhaustive strategy for a University it will be necessary to firstly record its strongest points and identify those that call for improvement, subsequently these should be counterchecked against threats and opportunities present in its field of action, determine how its "competitors" operate and what services will be worth adopting so as to attain its goals (SWOT ANALYSIS). Thus, we reach to the formulation of a strategic map outlining the goals of each dimension of the Academic Scorecard and the required metrics for the attainment of these goals. In order to identify the goals and indicators most suitable to the particular targeting of each University, it is often necessary to make use of widely used strategic management tools which are applied individually or/and in combination, according to each case, such as PVA, QFD, ABC & AHP or ANP etc.
Let us note at this point that the key for the successful application of BSc is the appropriate selection of the indicators and their correct weighting so as to determine the extent to which each indicator effects the overall strategic goal. This particular aspect is totally ignored by Greek Higher Education Institutions while the overseeing authority (HAHE) does not provide any relevant guidance within the context of the formers' independence.
By applying the Analytic Hierarchy Process in the development of the Balanced Scorecard this goal can to a great extent be attained. The application takes place in two stages, during the first one the most important Key Indicators are chosen while in the second stage the weighting factors of the indicators participating in the development of the BSc are determined [16]. In particular: During the First Stage the most important Key-Indicators are selected, these are determined through the application of methods such as SWOT Analysis, Quality Functioning Deployment and Product Value Analysis etc. As it was previously mentioned, in the Second Stage the statistical weights of the indicators applied in the BSc will be calculated. This process takes place in two levels. The first Level concerns the hierarchization of the four basic aspects of the BSc and the designation of the coefficients with which the aspects will be included in the final table. While the second Level concerns the calculation of the statistical weighting of the Key-Indicators of each aspect of the BSc independently. The AHP method predisposes that the factors that are present in the hierarchical structure are independent from one another. Should this not be the case it is necessary to employ the analytic network process (ANP) [17]. Many decision related problems cannot be hierarchically structured since they include the interaction and dependence of the high-level factors in lower level factors [18]. ANP usually analyses the relevant weights of the performance indicators.
The methodology we propose "Holistic Process-based Methodology for the Management of Academic Strategy -From the Design to the revelation of the rationale of decision makers" contains the steps outlined below (see also Fig. 1 We deemed it necessary to take a process-oriented approach in describing our methodology by using the BMP software ADONIS Community Edition 3.0, so as to allow for the possibility to include the entirety of all its modeling aspects (who does what and with what necessary documentation, timing of processing each step, required human and material resources, involved IT systems etc.) and thus enable the analysis of our models as well as the implementation of their simulation for an academic term. Of course, to this day we do not possess all the parameters so as to be able to complete our modelling.
IV. AN ATTEMPT IMPLEMENTING BSC WITHIN THE GREEK CONTEXT As the bibliographic survey we undertook confirms, it becomes evident that parts of the methodology we proposed in the previous chapter have already been implemented as stand-alone applications and have been presented mostly as research papers in journals and conferences.
For this exact reason we have made the decision to focus on until recently unpublished (in Greece or abroad) aspects of our methodology. Subsequently we briefly refer to two recent paper of ours that come to fill research gaps. The first [19] relates to Greece and was presented during the CIE 2020 conference while the second concerns the combined application of BSC & Utastar [14] and is indeed considered as an important contribution as it comes to fill a research gap.
In particular, in [19] we attempt at managing the academic strategy of a Greek Higher Education Institution through the use of the Balanced Scorecard technique. The implementation of the aforementioned technique is performed with actual data. These resulted from the operational planning of a Greek University which following a series of meetings and consultations with its Quality Assurance Unit, Rectoral Authorities, the Deans of the faculties and the Presidents of the Academic Departments, specified the strategic goals of the University, defined the performance indicators and set the actions necessary for their attainment. Finally, they agreed on the desired academic performances for the following academic period. The abovementioned strategy along with its defining dimensions were approved by the Senate and were then forwarded to HAHE for the needs of certification of the Internal Quality Assurance System.
In developing all the stages and models of the academic performance scorecard we utilized the specialized software ADOSCORE 2.0 developed by the Austrian company BOC S.A. The software in question is considered to be among the best when it comes to applying the BSc technique and it is widely used especially in German speaking countries. By using ADOSCORE 2.0 we followed the classic modelling and documentation generation methodology.
While being aware that this attempt does not constituteespecially in our times -an extraordinary research based contribution at an international level (taking into account similar efforts by foreign Universities), it is worth mentioning that no similar endeavor has ever been applied or published to this day in any Greek University. Therefore, through this chapter of our paper we present an innovative approach for Greece as well as a sample of empirical application of the theory which can be easily adopted.
In Fig 2 we present one of the most important modelling diagrams of our endeavor, which is the Cause and Effect Model of the Greek University in Attica. In this model we can monitor all strategic goals of the institution and the performance indicators that measure them in the four dimensions of its BSc. In our other work [14], we attempt to apply the last part of our proposed methodology regarding the combination of the BSc technique with the aggregation disaggregation theory of the Multiple Criteria Analysis. Specifically, we used the UTASTAR algorithm [21], [22] in order to reveal the cognitive style and the behavior of the decision maker whose task was to assess and rank the strategic performance of seven Greek universities. For the evaluation of the strategic performance we used 26 key performance indicators that are developed and proposed by the Hellenic Quality Assurance and Accreditation Agency.
In the following paragraphs we present the main outcomes of the application of UTASTAR algorithm which concern the definition of the criteria Weights (Fig 3) and the performance of the kpis in each dimension of the BSc (Fig  4).
The definition of the criteria weights will reveal which key performance indicator of each dimension are most important for the decision maker for the assessing the academic strategic performance and thus for implementing educational policies. As we can see on Fig 3, the most important criterion for the decision maker regarding the education dimension is the average number of students per undergraduate study program, following by the scholarships that is important kpi because it can affect positively other dimensions of the BSs, and the studies duration that consumes economic budget from the university affecting its financial performance. On the other hand, criteria like the number of Erasmus students and the degree grades are not crucial for the decision maker.
In the personnel dimension, the key performance indicator that plays the most important role according the preferences of the decision maker, is the criterion of professors' ratio per student and second the professors' ratio per undergraduate study programme. These two criteria can be considered as crucial factors for a university regarding his productivity in terms of academic excellence and competitiveness among other institutions. The other criteria of this dimension seem to be not so important for the decision maker.
In the other two dimensions, a brief analysis of Fig 3 depicts that the number of PhDs and the number of citations for the research dimension and funding through ESPA for the financial dimension are the most important factors for the decision maker in order to assess and further rank the academic performance of the universities and more over to draw strategic actions.
As we mentioned before, another useful outcome of the algorithm is that it depicts the performance of kpis in each dimension of the BSc (Fig 4). At this point we must clarify, that the performance scores are normalised between 0 and 1. As we can see in Figure 4 in the education dimension, the University 4 has the best performance among all Universities following by the University 2. Thus, the other Universities like Univ. 3 and Univ. 6 should take, according the preferences of the decision maker, all necessary actions in order to improve the performance of the kpis that belong to this dimension of the balanced scored.
At the personnel dimension, the University 2 and the University 6 have the best performance since they perform better in criteria's like professor's ratio per student and professors ration per undergraduate study program. On the other hand we observe the worst performance at University 7 and University 3.
Analysing the research dimension we observe that the best performance have the Universities 5, 4 and 2, while the other examined universities should design and implement policies in fields like quality of research, research expenditure, and mainly attract more PhD students in order to improve their position.
Finally, the last dimension that plays an important role in the implementation of the strategy is the financial dimension where it's obvious that the best performance is observed in University 1, University 3 and University 4. Obviously, the decision maker attributes high performance to these Universities because in criteria like ESPA funding they perform better.
Through the for the first time combined application of the BSC & Utastar technique using an actual case study from the Greek Universities domain, we managed to reveal the cognitive and the behavior style of the academic strategy decision maker. Hence, the special value of this methodology is that it can be used by any Quality Assurance Unit or evaluation authority for the academic classification of departments of Universities or between Universities.

V. CONCLUSION
In this paper we posed the problematic of the holistic management of an academic strategy and its handling techniques. The systematic bibliographic surveys conducted showed that the BSC as well as a multitude of MCDA algorithm forms constitute the most important tools for the holistic management of an academic strategy. Additionally, the specific elements and complexities in applying them were revealed. Based on this experience we proposed a holistic process-based methodology for the management of an academic strategy which spans from its design and oversight, to interpretation issues of the academic classification of departments of Universities or between Universities where assembly bodies (Quality Assurance Unit, HAHE) are active. It was found that a research gap exists when it comes to the issue of interpreting academic classification when sufficient documentation is lacking, and the classification is primarily reached empirically. This particular issue was tackled through a combined application of the BSC & Utastar techniques using an actual case study from the Greek Universities domain. Since this case should be viewed as the rule of thumb for most Universities (the application of numerous MCDA steps is considered a