Research data repository requirements: A case study from universities in North Macedonia

3.1 Methodology Design

The focus of this work was the RDM user practices and requirements at higher-education institutions in North Macedonia and their matching to IR services or features. Reflecting this in our methodology, we adopted the survey as the approach of choice. To address this focus, we needed to reach a large number of participants, and gather information both in breadth (reach different institutions), and depth (elicit enough details to understand their practices). We designed the questionnaire with this in mind.

As an additional input, we considered existing recommendations about IR services and features from relevant bodies (RDA, for example) and few of the works cited in the “Related work”, as a means to frame the survey. This was in terms of both questions and the available options to those questions that participants chose from (for the close-ended questions). In our case, where a preliminary study of the RDM practices in universities in North Macedonia was not available, compiling the questionnaire with this input proved valuable. Figure 1 shows these different, at times complementary considerations when specifying our methodology, which proved suitable to our target group’s specifics.

Fig. 1.

Methodology components

This methodology clarifies the study context - both based on previous work and past and ongoing initiatives - for IR and RDM practices, and the means to address such requirements for the universities in North Macedonia.

3.2 Survey Design and Administration

For the survey, we prepared a questionnaire consisting of 20 questions and shared it with the university teaching and research staff, ensuring we covered all the departments. For it, we mainly relied on closed-ended questions, with few exceptions where participants, by choosing the predefined option (“Other”), could provide a free-text answer. These exceptions are especially important for cases where providing a final set of options to choose from is not practical or exhaustive.

The questionnaire consisted of five main sections (see Table 1). In the first section the participant demographic was gathered, such as their academic engagement experience, role (teaching, research, or both), department they belong to, etc. The next three sections of the survey aimed at understanding user practices in relation to different RD lifecycle phases, such as data creation, preservation, and access, correspondingly, whereas the last section focused on participants’ understanding of and feedback (and preferences) concerning the features that an IR should provide. In the final section, we also asked participants to (optionally) provide their email, in case they are willing to participate in a qualitative information gathering follow up in the future. In addition to the main survey sections, we also included questions that provided information to introduce the survey topics, gather participants’ email addresses, as well as explain the survey aims, along with a request to get the participation consent. The survey was anonymous and confidential, meaning that their identity is not known, unless they opted to be available for a follow up via email. In order to validate our survey, we took into account the work from Guindon & Dennie (2016) and that of Hsu (2016) regarding researcher practice and repository functionalities to finalize our survey.

Table 1.

Questionnaire: Categories and Questions (Limani et al., 2020)

Our target group included university staff. In the context of administering the survey, we had to account for a time when they would be relatively free, such as outside of lecture, exam or grading sessions. Our study included two such phases, and for both phases we distributed the survey at the end of the semester, before their summer break, or before their semester started (with lectures), which makes for ideal periods to engage the academic staff in activities such as the survey. In any case, we were aware of the potentially low number of responses typical of surveys (as witnessed during our few distribution attempts throughout the process). Moreover, a lot of part-time research staff are usually harder to reach in such cases, depending on how often they check the official university email account, the period of their engagement at the university (certain semester, for example), and so on.

When collecting practices from one university to that of the national level, we wanted to take an incremental approach. As we experienced with our previous work with a single institution, an incremental approach helps conduct the right message and collect the right feedback, especially since the topic of RDM might be relatively new for them. For this reason, we selected three national universities – potentially with higher chances of being engaged with RD – to survey about RDM practices and their mapping to features of an IR.

The data collection was conducted twice by sending an email invitation to university staff to take part in the survey administered in Google Forms. The first instance was during March - April of 2019 when we collected data from the South East European University. Details of this process are documented in a previous study (Limani et al, 2020). The second instance was during August - November of 2019 when we collected data from two other universities in North Macedonia, namely, Ss. Cyril and Methodius University (UKIM) and Goce Delcev University (GDU). For the former, our responders were exclusively from the Faculty of Computer Science and Engineering (FINKI). For the purposes of completing the study, we relied on fast responsiveness of this department and mostly used this data for their descriptive significance. This university, however, is the biggest one in the country, as well as the largest producer of research data across domains, such as technical, social, medical, and so on. In both instances, participants included teaching and research staff as well as full- and part-time engagements.

Reminder emails are a common practice when conducting surveys, as they help participants not lose sight of the survey invitation, thus improving its completion rate. In this perspective, our survey was no different, and we sent a few email reminders to participants about the survey. After the last such reminder, the response rate for each university was as follows: SEEU - 45%, UKIM - 11%, and GDU - 23%. Table 2 shows the participants’ teaching or research experience, and their type of engagement at the university for the three institutions. Considering that our intent with this study was not to compare between different universities, but learn from their practices, we maintained that such discrepancies in the number of responders among universities is irrelevant to the study objectives.

Table 2.

Characteristics of survey respondents

It is worth noting that the surveyed institutions are at relatively different stages between and across them. While some of them have established (public) IR for research publications, others have it restricted to institutional use. In any case, this is a good indication of established standards for research publications management, which gives us confidence in the usefulness of our study focusing on IR features for research data management. Finally, knowing the local circumstances, we are aware that a lot of RD is produced in specific fields of research, such as medicine, biology, chemistry from one side, and (other types of data) social sciences, ethnography, archeology, history, music, etc., from the other side.

4.1 Data storing formats

One question on the survey asked participants to select the kind of format they use to save their research data. Participants could select more than one format. We grouped participants by their field of research in order to see any differences between various fields. We relied on the FRASCATI⁷⁾ categorisation provided by the Organization for Economic Co-operation and Development (OECD). FRASCATI covers six categories, namely: Natural Sciences, Engineering and Technology, Medical and Health Sciences, Agricultural Sciences, Social Sciences and Humanities. Our participants belonged to all areas except Engineering and Technology.

In Figure 2 we can see that Natural Science researchers save their research data mostly in spreadsheet formats (Excel, OpenOffice Calc, and Mac iWork), whereas researchers from Social Science save their data mostly as Text Documents (MS Word, OpenOffice Documents, and PDF). Detailed values for other fields (Medical and Health Sciences; Agricultural Sciences; and Humanities) and formats are shown in the figure. The group at the right end labeled as Unknown are respondents who did not provide their field of research or department in the questionnaire.

Fig. 2.

Research Data storing format by research field.

A careful observation of the graph shows that text documents and spreadsheets are the two most used formats across all research areas, with 83 and 70 respective counts. The least used are programming code and media with 27 and 24 respective counts. Somewhere in between lie databases and RD with 64 and 52 respective counts.

4.2 Data storage and backup practices

With two of the questions, we were interested to learn where researchers save and back up their research data. As seen in Figures 3 and 4, most of the participants prefer to store and backup their research data on their computers including portable storage and flash drive. Essentially, participants exhibit similar patterns - use the same devices - to store and backup their research data. Their second choice is cloud storage, which includes Dropbox, Google Drive, MS FileShare, etc.; the third is Research Repositories, whereas the final choice represents the use of department or laboratory servers.

Fig. 3.

Location preference when storing research data.

Fig. 4.

Location preference when backing up research data.

We see the same patterns for both storage and backup, that is, users demonstrate the same choices in terms of the medium used to store and backup RD. Another interesting observation is that of how many of the participants adopt each practice (storage and backup). It may come as a surprise that more of them (179) backup their RD than they store (171), but this could be due to the multiple choices available for these questions. In this case, chances are that a user uses a single (with exceptions) medium to store, and more than one (at least in few of the cases) to back up the RD.

4.3 Data keeping policies

Most participants indicate that they do not have a regular period of collecting research data, but they do it on a project basis. Some participants collect data monthly, some yearly, whereas a small number does it weekly (Figure 5). On the other hand, most participants, as shown in Figure 6, do not have a strategy on data storage/archiving after their research activity or project completes. A large number of participants prefer to keep that data three-to-five years after project completion. Similarly, a large number of participants prefers to keep the data indefinitely, whereas a minority keeps their research data up to one year after the project completes.

Fig. 5.

Frequency of research data creation/collection.

Fig. 6.

Period of keeping the data after project completion.

A key element for this theme is that a research project seems to dictate participants’ RD creation or collection, and storage timeline patterns. While this is clear from Figure. 5, Figure. 6 seems to show a set timeline for keeping the RD for what could be common project timespan (both for the “3 to 5 years” and “up to 1 year” options).

4.4 Data sharing

Researchers who collect research data typically have a need to also share those data with collaborators. The survey results indicate that almost all participants share their data. Participants, however, differ greatly about when they share their data. As shown in Figure. 7, most of the participants share their data after project completion, whereas a considerable group does it upon request from their collaborators or people interested in such data. Similarly, a large number of participants share their data immediately after it is collected, whereas a minority shares it during its creation.

Fig. 7.

The period when participants prefer to share their data.

It is interesting to find that most participants share their data as an attachment via emails. Another large proportion of participants prefers to share their data using a portable storage or cloud storage, whereas a minority prefers to do it using the department server or data repositories. Figure. 8 shows all the data sharing practices from the survey.

Fig. 8.

Preference on how participants share their data.

Similarly to the case of RD creation and preservation discussed in the previous theme, we can infer that RDM policies on when to share RD as well as the mechanism to do it seem to be missing. Not only was this not part of the responses, but no one from the participants mentioned it as a support to their preferences - be it the moment or the means of sharing RD.

4.5 Viewpoint on research repositories

Of the 110 total participants, a majority of them (78) have no awareness of the existence of any research data repositories (Figure. 9). Despite this, most participants express interest - said either yes or maybe when answering the survey - in trying such a service, thus demonstrating a need for it. Just a fraction of participants indicated no need for such a service (Figure. 10 shows the details).

Fig. 9.

Participant awareness of existing research data repositories.

Fig. 10.

The need for research data repositories.

A final piece of information that the survey could not capture and thus could not be subject of analysis for this theme relates to explaining the hesitation of those 53, and especially the refusal of those 4 respondents. What could cause these responses is perhaps lack of information about data repositories, a previous experience showing such repositories as unsuitable for their RDM needs, or disciplinary practices without a corresponding match in an IR feature, could have hampered IR adoption.

4.6 Data repository features to be supported

We were interested to learn what features of research repositories our participants considered important when managing their data. Figure 11 lists six features with storing data and metadata being the highest and creating metadata and documentation and archiving data being the lowest in terms of features. We were particularly interested in knowing the participants’ attitude towards the access control feature.

Fig. 11.

Research repository features ranked by participant preference.

As indicated in Figure 12, the access control feature is preferred by a majority of participants, whereas the rest were either not sure or did not need such a feature. This also reflects the different types of RD that participants create, for which a “fully open” or “fully closed” access control mechanism surely is not (“expressive”) enough.

Fig. 12.

The importance of access control.

These findings reveal interesting practices by our participants. It is a positive trend to see that RDs are present - generated, published, reused, etc. in a variety of formats and to a different extent (Figure. 2). Despite the fact that most participants were not aware of RD and their features (Figure. 9), they expressed a very high interest in relying on them, emphasizing the ability to store and distribute data and metadata (Figure. 11), or even the need for different access controls during the distribution of the same (Figure. 12). As seen in Figure. 3, IRs are the lowest used means for data storage and backup. This is in line with the findings shown in Figure. 6, which show that participants lack awareness about IRs, although they see them as a potential support tools for RDM activities.

5.1 RDM practice: Discussions

With the analysis complete, we now discuss the identified RDM practices, including every theme, in relation to recent efforts to identify RDM practices and map them to IR features. The motivation to adopt IRs was one of the aspects we explored in this paper. In this context, similar to Asadi et al. (2019) and Kipnis et al. (2019), our study participants acknowledged the benefits of IRs. Despite the low awareness of research repositories, survey participants in our study expressed the need for an IR as means to support their RDM requirements at a university setting.

The textual, databases, and spreadsheets remain the most prominent RD types, which is similar to what Guindon & Dennie (2017) report. Additionally, practices such as storage (local, external, or cloud storage), and dissemination (via email, portable or cloud solutions) are based on similar means. On a final note, 42% of respondents of the Guindon & Dennie (2017) study are not aware of data repositories, which does not seem too different from the university staff in North Macedonia. This just goes to show that RD usage is gaining momentum at a global scale, regardless of the minor details of its adoption with different researchers and regions. One aspect that Guindon & Dennie (2017) cite is the lack of incentives around RD, something that Hahnel et al. (2020) also present. The latter study presents some interesting findings about motivations and obstacles of sharing (open) data. Namely, survey participants of their study generally feel that sharing the data is not valued enough. Moreover, data citation, increased impact of a research work, and co-authorship on publications are the top-3 reasons to share, whereas concerns about misuse of data, sharing sensitive data, and lack of credit or acknowledgment are the top-3 reasons not to share the research data. Motivations for RD sharing and, generally, handling remains in our interest - and future work - to study with our participants.

Based on the data analysis in section 4, the need for organized storing, sharing and better management of the data is obvious. This is in line with (a subset of) the categories that Hsu (2016) or the Repository Platforms for Research Data IG have identified. On that note, sharing RD via email is the most popular approach, which, considering its capacity, could be an indication that the amount of data being collected and shared by those participants is relatively small in size. Of course, there remains a portion of respondents (57% of the cases) that we assume work with larger RD sizes, as noted by the other means of sharing it, as discussed in Section 4.4. Since we included staff members across university departments, this was expected as it shows the different practices across domains in terms of RDM.

What do the storage and backup practices show? In our view, we see room for improvement in moving from existing means, especially the local ones (almost 60% of the participants practice this approach; see Figure 3), towards more institutional, data repository-based solutions. Thus, although a considerable majority have stated that they currently store and backup their data locally (section 4.2), almost everyone expresses the need to share in different periods of time (see Section 4.4). Consequentl y, the use of an appropriate repository would simplify and facilitate this process, as opposed to current email distribution practices, external portable storage, etc. This need is further reinforced by the need to store and preserve the data for a longer period of time, as presented in Figure. 6.

On another note, IRs provide an appropriate path towards data FAIRness. Namely, considering the variety of RD and research practices across disciplines, de León and de Ferrer (2019) provide recommendations that RD repositories could adopt to become more FAIR. The mere adoption of IRs provides a good starting point towards the adoption of FAIR principles. As pointed out by Burns et al. (2013), two aspects that IRs adoption brings can be seen in the “discoverability of work through robust metadata and providing a permanent URI for that work”, which map well with some of the FAIR criteria. The aspect of FAIR, although not explicitly explored in our work, seems to align well with the participants’ practices (as shown in Figure. 10) since most of them either want or could try RDM services as supported by an IR.

We would like to complete this discussion with a view on RDM policy, which represents a common thread of the identified RD practices in our survey. While considering the answers from the questionnaire in Table 1, we did not come across any reference or even a hint of an RDM policy being the reason for the practices we identified. In this way, how long a researcher keeps their data, or how they share it, or when, to name but a few, are not rooted on an institutional RDM policy, but on, what we can assume, individual practices. We believe that an RD policy, coupled with incentives for complying with it (staff evaluation, promotion, institutional standing, etc.), could impact what RD practices researchers adopt.

5.2 RDM Requirements

The recommendations from the RD repository bodies, the related work on the topic, as well as our survey, provided a holistic context to understand RDM practices in the broader context, requirements that stem from them, as well as the more narrowed focus of RDM requirements for academic institutions. In this part, we present the (functional) requirements we identified as valuable or even explicitly needed for the survey participants.

The survey had a dedicated section on IR features that the participants would want to use for their RDM activity (see “Research data repository requirements” in Table 1), meant to both help us understand researchers’ needs for IRs, and serve as a guide to universities or other bodies and institutions to potentially close such a gap and invest in providing more RDM support. Please note that, to the best of our knowledge, this is the first such study of RD management practices and their mapping to IR features for universities in North Macedonia.

Survey participants were interested in different IR features, and these features could support activities in many phases of the data lifecycle. In order to discuss their interest through a data lifecycle perspective, we chose the model from the UK Data Archive8), among many others. As it can be seen from Table 3, the presence of data activities in the different phases is different, including a case (Re-using data) for which we did not record any activity in the survey data. This is not to worry, however, since we were not focused on evaluating the adoption of a data lifecycle as means to organize the RD practices at the participating higher education institution staff members. With that said, let’s next discuss the IR requirements through the data lifecycle perspective:

Table 3.

IR requirements categorized based on data lifecycle

- Collecting data. In this context, participants were interested in a feature that would allow them to create meta(data) and accompanying documentation (in 51 of the participants).
- Publishing and Sharing data. Once meta(data) is there, publishing and sharing it becomes an important next step. We see this with 60 of the participants when they specify distributing, sharing and promoting as a feature they need in an IR. At this point we would like to point out one requirement in particular that we identified and that is relevant to this feature - that of access control. Two thirds of the participants explicitly were in favor of a feature that enables them to publish and share RD based on a variety of access levels. However, considering that 21 were keen on publishing their RD with full access, while 20 of them were not sure, these two options can also be seen as just additional access control levels. Access control is important as it enables a research team to provide different access levels to a RD collection. For example, one can provide a different (re-use) access within the same research group, department, or institution; the (outer) community, researchers at the national level, and so on.
- Preserving data. Most of the requirements we identified are in relation to data preservation tasks. In this context, 63 participants selected storage, 60 selected backup, and 51 selected archive as features in an IR. When it comes to backing up RD, users prefer having different considerations to do so, such as based on a schedule, mode (ad hoc vs a more general approach), etc. Also, please note that there is a slight difference between the need to create and store meta(data): generally, more researchers store (download for reuse) rather than create RD themselves.

As discussed in 4.5, the survey participants generally are not familiar with IRs and their capabilities, but those that do, indicate a readiness to use such a solution. Considering a(ny) RD life cycle as a categorization framework, the IR requirements we identified in our survey are part of many life cycle phases (collect, publish, preserve), echoed to a different extent (some were more referred to than others in the replies) among the participants. As we pointed out, while storing meta(data) is the highest sought out feature, there is a relatively small difference with the remaining four IR features selected by the survey participants. Moreover, the requirements do not reflect any specifics of a field or domain the participants adhere to, and rather represent some of the more common IR features, generalizable at an institutional setting, in this case a university.

5.3 RDM practice: Recommendations

The methodology we adopted for this work enabled us to have considerable insights in RDM practices and IRs as means of supporting such practices. We would like to next share some of this insight, which could be used to explore RDM practices with new or existing academic communities, and perhaps other institutions.

The role and valuing of RD. The research staff would be more keen on focusing on RD as an equally valued deliverable as research publications. Similar to the incentives for researchers to publish, and the accompanying infrastructure - including publication repositories, RD could see an uplift if university staff members, as well as other relevant bodies and institutions, are also valued based on the RD they collect, maintain, publish, share, and so on.
Research groups are ready for RD sharing. We speculate such different strategies are dependent on the level of collaborators involved in the project. If the project has many collaborators, participants have the need to share those data during creation or immediately after the data have been created, perhaps, so that collaborators either complement the data creation process or they use it for analysis. On the other hand, researchers who do not have many collaborators in their project share their data after the project completion or upon request. It seems that most researchers belong to the latter group. One recommendation would be to adopt RDM practices (let’s say from “Publication and Sharing”) with researchers that are part of a project and for whom sharing RD is more of a need than with individual researchers.
Start with basic activities from the data life cycle phases. In the previous sub-section, we listed the RDM requirements we identified in this study. They represent some of the basic RD-related activities, such as creation, storage, backup, and so on. Seeing that these requirements appear across different university departments implies that they are common enough to be considered as an initial path towards pushing for a broader adoption of RD within research practices at universities.
RDM policy. There are many facets of RDM practices that could impact the culture of RD. Some of the findings in our survey go in the direction of supporting research data management policies. Any such policy initiative - be it at an institutional, broader, or national level - should strive to support the incentives and lessen the eventual barriers as Griffiths (2009) identified in the survey practices. In this context, there seemed to be a lack of such policies at the institutions we surveyed, as there was no rationale given based on institutional RD policies for any of the practices reported by the participants. In a way, this is an umbrella (catch all) for other recommendations that an institution could adopt, and one which enables an institution to pace their RD-related plans according to their goals, available resources, research culture, etc.

5.4 Methodology limitations

After discussing the results and our findings, we would like to briefly discuss the limitations of the methodology we adopted, and provide the rationale for them.

Some of the limitations we see in of our methodology include:

- The topic scope. Being that it is one of the first surveys that focuses on RDM practices for universities in North Macedonia, we had to decide on its scope - its breadth and depth. The former would include the spectrum of RD life cycle, and its corresponding phases, and the latter would include the level of details for each such phase, i.e. activities around RD that go with it. As an initial exploration, we wanted to start “small”, i.e. be as close to the targeted participants as possible about their RDM activities. The risk of increasing either the breadth or depth would result in no feedback, letting the participants potentially feel not up to the task or even wrongfully invited to fill out the survey.
- Target group. We targeted three universities for the survey. Even though they constitute a considerable part of the academic staff in North Macedonia, we cannot confidently report on RDM practices at the national level. We were aware of this since the very beginning of our exploration of RDM practices in academic institutions, and we wanted to have a smaller scope, and incrementally include more institutions along the way. To maintain this, we chose three of the larger universities in the country, a target group whose RDM practices were manageable for our research scope. Finally, even for these three institutions, not all of the invited staff members participated, which leaves us with limited knowledge of their RDM practices.
- Data analysis. Due to the number of participants, the research scope, as well as the topic, which is relatively new for our target group, the data analysis was of a descriptive nature. Moreover, it only includes the practices of this specific target group (the three universities), and only of the participants that responded to the survey. As such, the analysis could not be used to predict RDM practices or potential IR features based on the data we gathered. Thus, it cannot be generalized to broader groups - academic staff - and only depicts the practice of 110 survey participants.
- Ethical aspects. When collecting and sharing data, inevitably ethical aspects become highly relevant, such as complying to the General Data Protection Regulation (GDPR), for example. Such aspects should be reflected within IR features and even help researchers easily adopt ethical ways of dealing with the data they collect and share, whether those data belong to private individuals or companies. Considering the complexity of the topic, we do not address these in this study, however, we recognize their relevance.

Lastly, we would like to mention generalizing the results of this study. Namely, since we do not have representative samples from all the departments surveyed, thus their corresponding fields, these results do not reflect the RDM practices for those departments (and corresponding fields) at the national level. While we were able to cover different departments, we could not control a representative response rate from each department. For example, having more feedback from one department or field over the others will necessarily reflect the corresponding practices in the results. This, in turn, would make it seem as these practices are common at the study level, which could not necessarily hold.

A final comment to wrap up this section goes to the lack of an institutional policy for RD. We notice this from the feedback, i.e. the parts where survey participants do not mention any policy as a rationale for their RDM practices. We strongly believe that a policy that promotes and values RD-related tasks - their creation/collection, curation, sharing, publication, etc. - would benefit the research staff at universities, as it would provide guidance and motivation for them to engage with RD similarly to how they engage and treat research publications.