해상교통안전진단제도의 메타평가 모형 개발 및 적용에 관한 연구

Abstract

This study is intended to comprehensively analyze and evaluate the Maritime Traffic Safety Scheme(MTSA Scheme) conducted in Korea up to the present by developing the Metaevaluation model to be applicable to this safety assessment. For this purpose, the Metaevaluation model is designed for the maritime traffic safety assessment, developing evaluation index which is based on the Metaevaluation model to be verified, thus having it practicable for the MTSA Scheme.

In the process of illuminating the essence of maritime traffic safety assessment, it is clarified as preevaluation and institutional evaluation. Although it can be identified as a type of the policy evaluation in a broad sense, it has its own unique analysis.

As a result of reviewing the previous study about Metaevaluation, Metaevaluation is a series of task to investigate the validity for the initial evaluation from the perspective of the supervisor, including the overall evaluation system as well as the performance of the evaluation. Accordingly, it works as a tool to examine whether to utilize the outcome of the evaluation or not, it is also regarded as a comprehensive metaevaluation to ensure its value and advantages.

Metaevaluation of the basic model has been derived from the standpoint of public policy, Metaevaluation model of MTSA, PIP'OU evaluation model was set up based on this, it can be identified as Plan, Input, Process, Output and Utilization of the cyclic process. This accounts for the Metaevaluation in the terms of a systematic opinion, allowing us to be more accessible to its own purpose of improving the evaluation system via the reflux of the evaluation result.

For the main focus, it is provided that assessment grounds, goals, target, and design entry to the domain of assessment plan. assessment capabilities, budget, data and period items are offered to the domain of the assessment input. Assessment perform methods, content, items, procedures items are offered to the domain of the assessment process. Assessment report reliability, usability, reporting results, results screening and follow-up management items are offered to the domain of assessment output. And finally assessment results management, diffusion and proposed assessment system to improve results are proposed by the domain of assessment utilization. 52 indexes of evaluation indicators have been developed for MTSA of the Metaevaluation.

Survey of the experts is used to inspect the validity of the metaevaluation model and evaluation index which were established by a theoretical discussion. Accordingly, the survey team of 20 members including MTSA of council members, specialty organizations, assessment experts conducted expert survery over two times. The result is that 44 evaluation index was confirmed by 12 indexes of assessment plan domain, 9 indexes of assessment input domain, 10 indexes of assessments process domain, 7 indexes of assessment output domain, and 6 indexes of assessment utilization domain.

In the following the verification for reliability was conducted targeting MTSA experts. A total of 30 questionnaires answered over the 0.60 Cronbach's α value in the evaluation domains and evaluation item, resulted higher overall reliability to(of) 0.967. Rating value of the importance for 44 evaluation indices was proved to be reasonable because it offers higher than 3.0 in each 44 index.

In a metaevaluation methods, evaluation scores were assessed the average of sub-evaluation index by 5-point scale as very high (5 points), excellent (4 points), usual (3 points), unsatisfied (2 points) and poor (1 point).

MTSA for the metaevaluation was carried out for the involved, and 121 questionnaires were analyzed using by Microsoft Excel and SPSS 20.0 program.

As the result of metaevaluation, it included grade of index, frequency analysis, difference analysis, correlation analysis, improvement priority, question analysis and case analysis.

Particularly, in the result of the analysis of the differences between groups, the difference of significance level of p <.05 appeared in three assessment domains including the assessment plan, input and output. And it is shown that the one of business operator and business approval authority is to be underrated by one grade comparing to one of the assessment agencies and review authority. This difference seems to be because business operator and business approval authority take account of maritime traffic safety assessment from the participatory perspective. The difference of significance level of p <.05 is unseen in the analysis for the average work period difference (One-Way ANOVA). It explains that there has been no differences between groups regardless of work period engaged in maritime traffic safety assessment.

This study suggests that the positive acceptance which is maintained by discourse model has to be found by means of the effective model of MTSA in order to enhance safety aspects and respond to trends of the change in the paradigm in MTSA.

In this study, the further implications can be found as follow

To clarify the nature of maritime traffic safety assessment, and seek to expand the research field about the meta-evaluation, and try the new approach to the research methodology of the meta-evaluation. Also the issue that is available for the policy is introduced in this study by presenting a model of the metaevaluation which was applicable to maritime traffic safety assessment.