Online First

First, in a study on the economic value of public libraries, Pyeon, Soon-hee (2006) analyzed the public library value measuring methods based on consumer surplus and the conditional value measuring method. As a result of measuring the direct economic value of the public library to the individual, 84% of the users said that the public library is influencing the individual’s life, and one person paid KRW 8.270 to use the library.

An exploratory meta analysis research of the study of the measurement of the library’s economic value was also conducted (Jung Hye-kyoung et al., 2009), and this study is based on the previous research and is based on six variables (survey point in time, library type, research method, scope of coverage, scope of service, per capita GDP), and six hypotheses were set for them. The results of the study show that consistent range of return on investment (ROI) is possible depending on the range of benefits (direct effect, indirect effect) and the level of GDP per capita. In the same context, Shim Won-shik (2010) examines the issues of this research from a theoretical and practical point of view at the time as the research on the economic value of library services is introduced in Korea in earnest. The results of the study show that there are difficulties in the measurement related to the selection of the service to be measured, the application of the conditional value measurement, the measurement of the monetary value against time, the processing of the anomaly, the application of the alternative, and the use of estimation since it is highly variable depending on the choice of measurement method, so it is necessary to gather some qualitative data for supplementation.

In a study on the economic value measurement cases of library collections and the latest trends, Pyo Soon-hee and Park So-yoon (2011) pointed out that the introduction of electronic information sources diversifies library collections and the need for user-centered valuation is increasingly growing. They analyzed the value of library collections and evaluation of economic feasibility based on library environment and presented basic data of user-centered valuation model for library collections. In addition, research on the economic value of university libraries has been conducted (Koh Young-man, Pyo Soon-hee, & Shim Won-shik, 2012). This study analyzes the appropriateness of various measurement factors and measurement methodologies that should be considered in the university library value measurement process, The purpose of this study was to explore the possibility of application to university libraries in the future. To this end, two university libraries were selected and one library was used to measure services such as loan, electronic academic information, reference service, user education, and space by CVM, and the other used the 3 measuring methods of time value, alternative service value, and CVM to measure the value of electronic academic information service.

A study of analyzing the economic value and influencing factors of children’s libraries was also conducted (Kang, Mi-hee, 2016), and parents of six children’s libraries were selected as a means of payment for each month, As a result of the analysis performed with the conditional value measuring method, the amount of the willingness to pay (WTP) was estimated to be 9,243 won per person per month. Factors influencing the amount of payment were the frequency of use, satisfaction, number of children, sex, age, and income. In addition, the effect by factor is more likely to be increased by the frequency of use (1.140), satisfaction (1.335), number of children (1.417), age (1.124), income (2.426), whereas for the gender, the likelihood of paying by women was lower than that of men as 0.364.

As interest in the benefits that libraries provide to users has increased, research on intangible library usage, benefits, and user benefits has been conducted. First, in a study on the measuring methods for the economic value of the use of public library (Pyo Soon-hee, & Jung Dong-yeol, 2008), the use value of public library is measured through four approaches: time value, substitute service surplus value, library service value, and library integrated value. As a result, it was found that the value of the user’s opportunity cost was 11 times higher than the cost value measured by the time value, and that the specific value was measured at a low rate of 0.3 through the amount of the user’s payment.

The results of this study are as follows. First, the characteristics of respondents in the measurement using CVM were analyzed (Pyo Suoon-hee, Koh Young-man, & Shim Won-shik, 2011) concerning the characteristics of the respondents and library, among others. As a result of the study, factors affecting the value of the payment which shows the value of the library are the usage type such as the demographic characteristics such as the user’s gender, income, age, income, and usage frequency. In the satisfaction and recognition areas, the satisfaction of librarians, the place of residents exchange, the degree of economic help, and the perception of applicants for arts and culture were influenced by the three types of services: information, facilities and programs. This study is considered a very meaningful study that comprehensively analyzes the impact on the measured value of all types of public libraries in Korea.

Therefater, the researcher Pyo Soon-hee (2014) conducted a study on the benefit evaluation of public library users, and reevaluates and interprets the benefits of public libraries revealed through foreign literature from the perspective of domestic library users. To this end, Pyo measured the benefits and level of library use for the S library users. The type and content of the benefits were extracted from the responses of the users freely described through the open questionnaire, and the level of benefits was identified through 13 benefit items based on the enjoyment index. The S library users recognized the benefits of library use in the order of reading, economic benefits, personal satisfaction, comfort, peace and satisfaction. As a result of the enjoyment result index, the user’s level of benefit was the highest in reading (4.31) and economic benefit (4.14). These benefits did not differ by gender and education, and the level of benefits varied by age and occupation. The benefits such as family ties, children’s reading, enjoyment of culture, and improvement of work ability differed by age group, and the economic benefits were found to be the most ineffective population such as housewives and job seekers.

In addition, a research on the economic value of library services has been carried out in the course of research on the economic value of libraries. Jung, Hye-kyoung (2005) proposed a new approach to valuation. The results of this study are summarized as follows: first, the validity and the applicability of the method are evaluated. The measurement factors and methodology are applied differently according to the type of resources (physical resources or human resources) used in the service. The development of the analytical model for the economic value evaluation of the specialized library service could be an opportunity to inform the true value of the specialized library because it is possible to objectively verify the value of the service.

Ryu Hee-young and Lee Doo-young (2006) investigated the economic value of Korean original information services and try to measure the economic value of original information services to determine whether it is worth for libraries to invest in the database construction. Conditional value measurement was applied to measure economic value. In order to measure the value of the original non-market information service, they selected a hypothetical scenario as a preliminary investigation, a review of experts, and a question method in order to increase the reliability of the questionnaire. As a result, the use value of 836 won per book was measured to be 236 won per year. The annual total economic value of the college students was calculated to be 83.18 billion won.

In addition, Shim Won-shik (2010) sought to examine the issues of this research from a theoretical and practical point of view, when the research on the economic value of library services was introduced into Korea in earnest. Specifically, the measurement difficulties associated with the selection of the service to be measured, the application of conditional value measurement, the measurement of monetary value versus time, the handling of anomalies, the application of alternatives, and the use of estimates were presented.

As a result of the research on the value of library service value, researchers Pyo Soon-hee and Cha Mi-kyoung (2016) conducted a study on the measurement of the economic value of library services in the National Assembly Library. The purpose of this study was to confirm the support of the stakeholders of the National Assembly Library by securing the economic feasibility of the service provided and raising the awareness of the users. To this end, the users of the National Assembly library were divided into internal users of the National Assembly, external users and users of the agreement institutions. Time value, alternative service value and WTP value measurement method were applied as a method for measuring value. If the users do not have the National Assembly Library and use other alternative services, the value of the additional time required for the year is 1,948.2 million won for internal users and 417,057.5 million won for external users, which is about 10 times the budget of the National Assembly Library. The total value of internal users by WTP was 8,528.3 million won, the total value of external users was 33,272.4 million won, and the total value of users of agreement institutions was 924 million won. The WTP value by user accounted for 77% of total value of external users compared to internal users, so external user value accounted for a large portion. The BC ratio of the National Assembly Library to the 2015 budget was 1:1.04.

Meanwhile, it can be seen that research has been carried out to prove the value of libraries much more actively overseas, which was published in 2007 by The Americans for the Libraries Council (ALC). In the report, “Worth Their Weight: An Assessment of the Evolving Field of Library Valuation,” they introduced an analysis method for the recently introduced economic and social effects, emphasizing the importance of representing value in the form of currency.

The Delphi method is a conditional repetitive questionnaire method, which is used to make decisions or predictions for adopting any of a large number of important questions. It is a method to organize the opinions of the expert group through the process of conducting the questionnaires to the selected group of experts, collecting the results of the questionnaires, informing the results of the collected questionnaires, and responding by referring to them. Expert groups are not only open to public, but also avoid the undesirable psychological effects of multiple domineering, influences of authoritative remarks, pre-coordinated group epidemiological weakness (Lee Jong-sung, 2001).

In this study, a specialist group was formed for the Delphi investigation, and experts in related fields were selected. Expert group consisted of experts who published research papers related to the value and influence of library or participated in discussions about the value of library, and professors and field librarians were appointed experts. In the case of on-site librarians, more than 10 years of experience were required for each kind of lecture. Because it is an evaluation index of the public library, it is necessary to consider whether the library librarian should be the only public librarian. The purpose of this study is to evaluate the social evaluation indices of public libraries. However, the evaluation index of public library developed in this study has a considerable part that can be applied to other kinds of matters, and it was necessary to discuss the validity of the index from diverse perspectives and angles.

A preliminary evaluation index to measure the value of the library was derived based on domestic and foreign prior research and case studies. The preliminary evaluation indices were surveyed by a group of specialists. After conducting the survey of each meeting, the evaluation factors were revised and supplemented through analysis, and finally, the social impact evaluation index of the library was developed (see <Figure 2>).

Fig. 1. 
The Delphi Investigation Procedures

For the reliability test, the Cronbach alpha coefficient was used. For the Delphi surveys, reliability can be measured using the generalization likelihood factor, which is the same as the Cronbach α coefficient (Lee Jong-sung, 2001). The content validity is to assess whether the test item reflects the content area to be measured. The content validity in this study was analyzed based on the content validity ratio (CVR) proposed by Lawshe (1975). The content validity ratio is judged to be content validity for the item when it is more than the minimum value according to the number of panels, and the criteria are provided in <Table 3>.

The validity of the Delphi method can be verified by analyzing the opinions (convergence degree) and the degree of consensus (consensus) on the items of the expert panel (Lee Jong-sung, 2001). The degree of convergence has a value of 0 when the result of the Delphi survey is converged at one point, and the value becomes large when the variance of opinion is large. On the other hand, in the case of consensus, the degree of consensus among respondents is verified. When the consensus is reached, it has a value of 1. In other words, the closer the convergence degree is to 0, the closer the agreement degree is to 1, the more consensus is reached on the opinions of experts. The convergence analysis is [(75th percentile - 25th percentile) / 2], and the agreement analysis is [1 - (75th percentile - 25th percentile) / median)] (Kang Yong-joo, 2008).

In the case of stability, it is considered that stability is secured when the agreement of responses is low due to the small difference in the survey responses of the panels in the repeated questionnaire process. This is measured by the coefficient of variation, and the analysis formula is [standard deviation / arithmetic average]. In general, when the coefficient of variation is less than 0.5, additional questionnaires are unnecessary, and when 0.5 to 0.8, relatively stable, and 0.8 or more, additional questionnaire is required (Noh Seung-yong, 2006).

In the case of the final evaluation factor selection, various criteria are applied according to the researchers. According to the research of Noh Seung-yong (2006), Seung Gi Yeol, Lee Shin-young (2011), Lee Min-hong, Lee Jae-jung (2010), Cha Seong-mi (2010), Seo Sung-yong (2010) et al. adopted an average score of 3.5 or higher and a standard deviation of less than 0.8 or less (Ahn Jin-sung, 2011), Lee Sang-sun (201) selected average of 4.0 or higher, Lee Young-soon has an average of 3.5 or more and a standard deviation of less than 0.8, while satisfying content validity, and Kim Jin-sun (2015) adopted a factor of 4.0 or more, or an average of 3.5 or higher and a standard deviation of 0.8 or less, while satisfying content validity. In this study, the final evaluation factor was adopted only for the items that satisfy the content validity, convergence degree, consensus, stability, average, and standard deviation shown in <Table 4> based on the third response (final questionnaire).

It is notable that research to evidence the library’s value has been conducted very actively. In the United States, the American Library Association (ALA) provides the analysis of the impact of libraries on local economies and communities. The comprehensive analysis of the research results shows that libraries have economic, social, educational, and cultural values, and they demonstrate their value through surveys of residents’ perceptions and methods of document analysis. The study area is very broad, including the United States, Canada and Korea, and the research results are very broad, with impacts on community building, recreation, health promotion, democratization, contribution to research, literacy and local economic value. <Table 5> summarizes the literatures related to various economic values carried out in Korea and abroad.

In this study, preliminary economic evaluation index was derived by analyzing the list presented in ALA and domestic research results and structuring core contents. The preliminary evaluation indices classified the economic value of the library into four evaluation areas: regional economic value improvement, economic revitalization, manpower development, job creation, and investment value improvement. The local economic value enhancement area was divided into two evaluation categories and nine evaluation indices were developed. In the area of economic activation, one evaluation item, three evaluation indices were developed. In the area of human resources development and job creation, two evaluation items and 14 evaluation indices were developed. In the investment value improvement area, two evaluation items and 10 evaluation indices were developed.

In the first Delphi survey, the suitability of the 4 preliminary evaluation areas, 7 evaluation items, and 36 evaluation indices derived from previous research was evaluated. In the case of the first questionnaire, the open-ended questions were included in consideration of the omission of the preliminary evaluation index derived from the previous study. The survey period was from January 4, 2017 through January 13, 2017. All 10 panelists who participated in the survey were analyzed using SPSS 21.0 in a sincere response. The results of the first Delphi survey are as follows.

First, the content validity is 0.62 for 10 respondents among the criteria of the content validity ratio proposed by Lawshe (1974). The results of the first Delphi survey showed that the indices with the content validity ratio of 0.62 or less were 5 in the area of economic value enhancement, 1 in the economic activation area, 10 in the human resource development and job creation area, and 8 in the investment value improvement area. In the first Delphi survey, 24 of the 36 evaluation indices were not satisfied with the content validity. On the other hand, in the content validity, the six evaluation indices were 1.00 and all respondents recognized that they were appropriate. Second, the degree of convergence and consensus was analyzed for the first Delphi survey validity test. The indices that exceeded the convergence index of 0.5 were four of the 36 items and the items with the agreement of less than 0.75 were the nine. Stability that confirms the consistency of the responses showed that all indices were very stable, below 0.8.

Third, in order to compensate the limitation of the simple questionnaire, the opinions of the experts were collected through the open questions for each evaluation area. “Library contributes to the economic activation of the residents through the resource sharing” was revised to “Library alleviates the economic burden of residents through resource sharing.” “Library’s social return rate is high” was revised to “Library’s social investment return value is high.” For the items that did not pass the consensus and the content validity and convergence degree, it was included in the second question item without deleting it considering that it is the first questionnaire. Consequently, there is no difference between the first and second questionnaire items.

In the second Delphi survey, the appropriateness was reevaluated in four evaluation areas, seven evaluation items, and 36 evaluation indices, reflecting the findings derived from the first Delphi survey. The results of the second Delphi survey were as follows.

First, the content validity was based on 0.62 as in the first Delphi survey. The results of the second Delphi survey showed that the indices with the content validity ratio of 0.62 or less were 5 in the area of regional economic value improvement, 7 in the human resource development and job creation area, and 6 in the area of investment value improvement. In the second Delphi survey, 18 of the 36 indices were not satisfied with the content validity, and the items agreed on the items compared to the first response (24 out of 36 evaluation factors). On the other hand, in case of content validity, 14 out of 36 evaluation indices were 1.00, and all respondents were satisfied. Particularly, in the case of economic activation evaluation area, all three evaluation indices showed that the content validity was 1.00.

Second, the degree of convergence and consensus was analyzed for the second Delphi survey validity test. The evaluation factors exceeding 0.5, which is the evaluation criterion of convergence, were two of the 36 evaluation indices, and the experts agreed with the first (four evaluation indices). The evaluation index, which is less than 0.75, has 7 expert evaluation points compared to the first (9 evaluation indices). As with the first questionnaire, the stability of confirming the consistency of responses was found to be very stable at 0.8 or lower in all items. Third, one evaluation index was revised for the third Delphi survey, reflecting expert opinions on the second Delphi survey. For the human resource development and job creation area, “Library contributes to the expansion of jobs throughout the community with direct (librarians, administrative, and other related personnel) and indirect job creation (publisher, retail and wholesale, DB vendors, printing companies, postal companies, food and beverage sales, and other management business, etc.).” Reflecting the expert opinion on the simplification of the index, the index was revised to “Library contributes to expanding job creation throughout the region through direct and indirect job creation.”

In the third Delphi survey, the suitability was reevaluated with 4 evaluation areas, 7 evaluation items, and 36 evaluation indices reflecting the findings derived from the second Delphi survey analysis. The results of the survey analysis are as follows.

First, the content validity was based on 0.62 as in the second Delphi survey. The results of the third Delphi survey showed that indices with a content validity ratio of 0.62 or less were 4 in the area of regional economic value improvement, five in the area of human resources development and job creation, and five in the area of investment value improvement. The items that did not satisfy the content validity in the third Delphi survey were 14 out of 36 total scores and agreed with items in the second response (18 out of 36 evaluation factors). On the other hand, in case of content validity, 18 of the 36 evaluation indices were 1.00 and all respondents were satisfied. Particularly, in the case of economic activation evaluation area, all three evaluation indices showed that the content validity was 1.00.Second, the degree of convergence and consensus was analyzed for the second Delphi survey validity test. There is no evaluation index exceeding 0.5, which is the convergence degree evaluation standard, and the opinion of experts is consistent with all evaluation indices. As for the evaluation index, which is less than 0.75, the consensus opinion was the same as the 5th evaluation index compared to the second (7 evaluation indices). As with the first questionnaire, the stability of confirming the consistency of responses was found to be very stable at 0.8 or lower in all items.

Third, in order to compensate the limitation of the simple questionnaire, the opinions of the experts were gathered through the open questions for each evaluation area. “Library contributes to the user’s willingness to pay.” was revised to “Library is so valuable that users are willing to pay for library services personally.” In addition, revision was made to “Library provides information and services related to job search, such as finding a job, finding a resource on how to write a resume, finding job information, etc.” and another revision was made to “Library is so valuable that users are willing to pay for library services personally.”

The evaluation of regional economic value improvement area consisted of two evaluation items and nine evaluation indices at the first. Finally, two evaluation items and five evaluation indices were derived through the third Delphi survey. Looking at the process through which the specific evaluation indices were derived, first, the index of “Library provides business and related research information, information needed for decision making, and investment information to those who work in the enterprise.” from the evaluation item of “Library helps local businesses grow.” together with the index of “Library analyzes and identifies the propensity of the firm and provides customized information about it.” for content validity and the index of “Library provides training and programs for start-ups and small businesses to reduce costs and reduce barriers to entry and business failures.” were deleted for their failure to satisfy content validity and consensus through the 1st, 2nd, and the 3rd.

Second, the index of “The consumption of library visitors improves the economic value of shopping malls around libraries (eg, restaurants, cafes, etc.).” for the evaluation item of “Library improves the economic value of the community.” were deleted for the failure to satisfy content validity at the 1st, 2nd, and the 3rd.

The evaluation of economic activation area consisted of one evaluation item and three evaluation indices at the first. Finally, one evaluation item and three evaluation indices were derived through the third Delphi survey. Looking at the process of deriving specific indices, it can be said that the initial indices remained unchanged since all experts showed relatively high agreement on the indices of the economic activation area. However, the content of the evaluation index reflects the expert opinions, and ‘Library contributes to the activation of the residents through the resource sharing’, was revised to ‘Library alleviates the economic burden of the residents through resource sharing’.

The evaluation of human resources development and job creation area consisted of 2 evaluation items and 14 evaluation indices at the first. Finally, two evaluation items and nine evaluation indices were derived through the third Delphi survey. Looking at the process of deriving specific indices, first, the index of “Library contributes to the expansion of jobs throughout the region through the creation of direct and indirect jobs.” for the evaluation item of “Library contributes to job creation.” was revised by reflecting expert opinions during the 2^nd Delphi survey, but content validity failed to satisfy at the 1st, 2nd, and the 3^rd and was deleted. The index of “Library supports the search for jobs, finding relevant information on how to write resumes, finding career information, submitting applications online, and getting training for jobs.” was revised to “Library provides information and services related to job search, such as finding a job, finding a resource on how to write a resume, finding job information.” Secondly, for the item of “Library contributes to the enhancement of economic capacity,” the four evaluation indices were not satisfied because content validity and consensus failed to satisfy at the 1st, 2nd, and the 3rd.

The evaluation of investment value improvement area consisted of 2 evaluation items and 10 evaluation indices at the first. Finally, two evaluation items and five evaluation indices were derived through the third Delphi survey. Looking at the process of deriving specific indices, first, the index of “The economic value of computers connected to the library’s Internet is high.” for the evaluation item of “Library’s social investment return value is high.” was deleted for the failure to satisfy content validity at the 1st, 2nd, and the 3rd.

Second, for the evaluation item of “Library contributes to the user’s willingness to pay.” all indices except “The higher the frequency of users’ usage, the higher the willingness to pay for libraries.” failed to satisfy content validity and consensus and were deleted. In addition, the index of “The higher the frequency of users’ usage, the higher the willingness to pay for libraries.” was revised by reflecting expert opinions to “Library is worth the user’s willingness to pay for the library services personally.”

In this study, the final evaluation index for the economic value of library was derived as a result of gathering the opinions of library and information science professors and librarians through three processes. Finally, 4 evaluation areas, 7 evaluation items, and 22 evaluation indices were derived. Specifically, two evaluation items and five evaluation indices were derived from the evaluation area of regional economic value enhancement, one evaluation item and three evaluation indices were derived from the economic activation evaluation area, and two evaluation items, and nine evaluation indices. In addition, two evaluation items and five evaluation indices were derived from the investment value enhancement evaluation area.