Restoration of Taikhar Rock Cultural Heritage Text

1. Introduction

Mongolia is home to many monuments adorned with ancient rock paintings. Petroglyphs, in particular, are valuable historical artifacts that narrate stories from the past. Researchers analyze the designs and techniques used in these carvings to estimate their creation periods.

Recently, the defacement and destruction of these rare and culturally significant monuments have become more prevalent. While some attribute this issue to local superstitions, others see it as acts of vandalism or intentional sabotage.

In Mongolia, 9,136 out of 82,096 historical and cultural immovable monuments were officially registered in last year’s census conducted by the USFM (2023). Among these:

• - 213 monuments were contaminated and damaged by fat.
• - 201 monuments were stained and otherwise harmed.
• - 263 monuments were carved, damaged, or destroyed due to human actions.

This highlights the urgent need for large-scale restoration and preservation efforts for these invaluable cultural treasures. This research will explore the possibility of restoring the inscriptions and carvings on Taikhar Rock, a popular tourist attraction (Ganchimeg, 2013).

The Tamir River has carved its way through the granite headland on its northern bank for millions of years, creating a striking gorge. Taikhar Stone, an 18-meter-high granite cliff (as shown in Fig. 1), stands at the edge of the North Tamir River, 22 km from Tsetserleg in the Ikhtamir Sum area of Arkhangai Province. This prominent landmark bears over 150 inscriptions in various scripts and languages, including Runic, Mongolian, Chinese, Uyghur, Sogdian, and Tibetan (NCCHM, 2020). Of these, Mongolian inscriptions (Ganchimeg, 2015) are the most numerous, with approximately 70 examples documented. The Turkic inscriptions on Taikhar Stone are notable for preserving the ancient Turkic alphabet, written in red and black ink. Taikhar Stone is classified as a historical and cultural immovable monument under the category of “Rock paintings and written monuments”. It was officially included in the “List of Historical and Cultural Immovable Monuments under State Protection” by Resolution No. 13 of 2020, issued by the Government of Mongolia (NCCHM, 2020).

Fig. 1.

The original image (nature) and color of Taikhar rock are highlighted

Recent acts of vandalism, such as graffiti overwriting (as shown in Fig. 2), have led to the degradation of historically significant cultural heritage monuments. Forged inscriptions and fake texts have further corrupted the authenticity of Taikhar Rock’s writings. These fabricated markings create significant challenges for researchers investigating the site’s cultural heritage. The presence of fake texts often causes confusion and misinterpretation of the original inscriptions. To address this issue, this paper proposes a method to differentiate between authentic cultural heritage texts and fake inscriptions on Taikhar Rock. The proposed approach uses CNNs and color image processing techniques to restore and enhance the original rock paintings and inscriptions.

Fig. 2.

A test image highlighting the original and color of Taikhar rock fake and cultural heritage text

2. Image Enhancement techniques

Image reconstruction techniques are selected based on the type and extent of damage in the image. The choice of method depends on factors such as the nature of the damage, available information on the degradation process, computational resources, and the desired quality of the reconstructed image (Ganbold, 2014). Using optimal techniques is essential for restoring the original image from a degraded or corrupted version.

Local contrast filters improve image details without altering the global contrast by applying an unsharp mask with a large blur radius. Histogram equalization is a technique that enhances contrast by redistributing the image’s histogram. It is particularly useful when the image has similar contrast values throughout, boosting overall contrast. This method works well for images where both the background and foreground are either bright or dark. CLAHE (Contrast Limited Adaptive Histogram Equalization), a variant of Adaptive Histogram Equalization (AHE), addresses the issue of contrast over-amplification (Pizer et al., 1987). It operates on small regions, or tiles, and combines them using bilinear interpolation to eliminate artificial boundaries. CLAHE enhances image contrast and can also be applied to color images. Adaptive Histogram Equalization (AHE) is a variant of the histogram equalization technique designed to enhance an image’s contrast, especially in regions with low contrast or uneven illumination (Sarkar et al., 2023; Bhat et al., 2014; Chen et al., 2023).

The key to effective image restoration is removing noise while preserving important image details. Old images often contain Gaussian noise (Ganchimeg, 2015), making Gaussian filtering an appropriate method (May et al., 2024). This method can be applied in both spatial and frequency domains. Therefore, we introduce Gaussian filtering for image denoising in our approach.

3. Convolutional Neural Networks (CNNs)

Machine learning technologies are known for their strong feature extraction capabilities, allowing them to derive complex high-level features from low-level ones. This makes them a more efficient and accurate solution for detecting damage to cultural heritage. In recent years, researchers have explored machine learning-based methods as a means of replacing traditional damage detection techniques for cultural heritage (Nazarian et al., 2018; Valero et al., 2019; Meng et al., 2021). These studies have targeted various types of cultural heritage, such as historical buildings (Adamopoulos et al., 2021), stone monuments (Hatir et al., 2021), glazed tiles (Chaiyasarn et al., 2018; Wang et al., 2020), wooden paintings (Angheluta al., 2020), and gold-leaf shedding on stone (Hou et al., 2024). To improve detection accuracy, different advanced machine learning algorithms have been employed, with convolutional neural networks (CNNs) being widely used (Chaiyasarn et al., 2018; Kwon et al., 2019; Wang et al., 2020; Zou et al., 2019; Hou et al., 2024). Convolutional Neural Networks (CNNs) can be used in various ways to detect rock art and ruins. A common approach is to train a CNNs to classify images as containing rock art or not, as demonstrated by Lobo (2023). This is done by providing the CNNs with a labeled dataset of images. CNNs can also detect faults or damage in rock images. CNNs have proven effective for detecting and identifying vandalism in rock art, as shown by Gkioxari et al. (2021). Region-Based Convolutional Neural Networks (R-CNNs) are specialized machine learning models for object detection in computer vision. As a result, Mask R-CNN naturally emerges as an intuitive extension of this approach, as noted by Hassan et al. (2022).

While there has been significant research on automatic detection of stone-based cultural heritage using computer vision, particularly deep learning methods, research on detecting Mongolian rock inscriptions or cultural texts remains unexplored. To improve the accuracy of graffiti classification tasks, data augmentation techniques are often applied. These techniques generate new images from existing ones, thereby increasing the dataset’s size and diversity while reducing the risk of overfitting.

4. Study area

To evaluate the proposed approach, we collected over 650 images of Taikhar rock. The Mongolian National Cultural Heritage Center created a 3D model of Taikhar rock from 374 photos captured by drones. This enables precise prediction of where cultural heritage inscriptions may be located. However, Taikhar rock contains many fake inscriptions, making it challenging to reconstruct the authentic historical texts. Additionally, historical records of the original inscriptions are extremely rare, leaving little to no comparable references.

In this collection, we aimed to cover a broad range of image types, including both printed and handwritten texts, with varying degrees of degradation and diverse content (such as different languages, artwork, and images). As with any historical image collection, this dataset includes a representative range of degradation issues. Table 1 provides details about the condition of the images used in this study. As shown, only a few images are in good condition (e.g., homogeneous background noise that does not affect readability). These images are useful for evaluating noise removal algorithms, as they are minimally impacted. The remaining images exhibit at least two issues, stemming either from degradation or the content of the historical text. The most common problems include uneven background illumination and mixed ink seepage. Since the test rock paintings have lost much of their color over time, the chosen image processing methods focus on enhancing local contrast and applying histogram equalization.

Table 1.

Details about the condition of the image collection used in this study (Total number of images is 650)

• ∙ Datasets: The dataset used in this study consists of 650 images of varying sizes, captured by drones and sourced from the internet.

5. Methodology

The image enhancement methodology was selected to significantly improve the clarity of blurred images, as the natural rock surface had shifted in color from brown to reddish and black tones. The culturally significant inscriptions were created using zoson painting techniques, including petroglyphs and ink paintings. The appearance of these inscriptions varied depending on their position on the rock and the interplay of light and shadow. Environmental factors such as direct sunlight, cloud cover, and lighting conditions introduced variations in the images, further influenced by the rock’s color and surface condition. Acquiring a high-quality original image is essential for effective restoration. This paper also presents methods for restoring and improving the defaced inscriptions on Taikhar Rock.

Fig. 3.

Methodology for enhancing color rock images

6. Results and Discussion

The use of Mask R-CNN for rock art detection offers several advantages:

• - As shown in Figure 4, it excels in accurately segmenting examples, achieving high object detection accuracy, and adapting to specific datasets. Its compatibility with pre-trained models on large datasets, such as COCO, enhances its reliability.

Fig. 4.

Some parts of the Mask R-CNN method experiment

• - Open-source community support and features like ROI Align further improve the model’s effectiveness in capturing intricate details of rock art.
• - When selecting and implementing a rock art detection model, it is essential to consider the dataset’s characteristics, available computational resources, and ethical implications.

Mask R-CNN has demonstrated its capability as a powerful tool for accurately identifying and analyzing rock art.

Conducted experiments using the Mask R-CNN model to evaluate (as shown in Fig 5 and Fig 6) its effectiveness in enhancing blurred images and petroglyphs. The experiments involved images from various rock art sites with differing levels of weathering and visibility. The results were compared with the original images to assess improvements in visibility and detail.

Fig. 5.

Training results of the Mask R-CNN method

Fig. 6.

Mask R-CNN method testing, detecting fake text and rock images

The Mask R-CNN R-101-FPN 3x model was used to detect inscriptions on Taikhar Rock (as shown in Fig. 7). Through R-CNN training, the model successfully recognized 95% of the texts on the rock paintings. An assessment of Taikhar Rock’s surface revealed that approximately one-third of the accessible area had been marked by human-made graffiti, while around 61% of the total area was affected by counterfeit inscriptions. The detection process, as illustrated in the images below, identified texts written in Mongolian Cyrillic as fake. However, cultural heritage inscriptions are often intermingled with forgeries, necessitating the distinction and restoration of authentic texts. To achieve this, the improvement of known cultural heritage inscriptions was essential, along with the creation of a database for further training. The primary aim of this research was to refine and enhance the clarity of cultural heritage inscriptions, as detailed in the subsequent test section.

Fig. 7.

R-CNN learning outcome Taikhar rock inscriptions

7. Experimental

A Pentax 645Z IR camera was used to uncover cultural heritage texts concealed beneath superimposed inscriptions. Additionally, an RTK multispectral Phantom 4 camera was employed to capture images of ancient oil painting inscriptions. While the extracted images revealed cultural texts, further enhancement was required due to the weathering and aging of the rock surface.

DStretch®, a plug-in for ImageJ© (Le Quellec et al., 2015), is specifically designed for enhancing digital images of pictographs and has been widely used globally for over a decade. However, it had not been applied to Mongolian rock art until this study, where we utilized it on the rock paintings at Taikhar Rock.

The experiments followed the procedure outlined in Figure 3 and summarized in Table 2. Table 2 presents part of an image from the Taikhar Rock cultural texts, along with the sequence of operations listed in the table header. The process included:

Table 2.

From the experimental results section

• (1) Denoising the original image.
• (2) Applying CLAHE histogram equalization.
• (3) Performing local contrast enhancement and applying a Gaussian blur mask (5x5 filter).
• (4) Reapplying a local contrast filter using a combination of mathematical arithmetic operators (LOG+XOR).
• (5) Applying a black mask to enhance color and changing the background to white.

These steps resulted in improved and clarified cultural heritage texts. Table 3 displays the outcomes achieved using the DStretch® plug-in.

Table 3.

From the experimental results section

Table 3 shows the test results of the original image and the images improved by our proposed method, which highlight colors with the DStretch plugin.

The DStretch plug-in produced suboptimal results in detailing image colors when the original image was of poor quality or noisy. However, with high-quality, low-noise images, the plugin effectively highlighted colors (Table 3). Therefore, when using DStretch to restore rock image colors, ensuring the quality of the original image is crucial. For low-quality images, pre-processing and enhancement are recommended before applying DStretch.

Experiments were conducted to filter the background and isolate the main text on a white background. The signal-to-noise ratio (SNR) was calculated to evaluate the enhancement. Table 3 demonstrates that applying the DStretch_lds plugin after improving the original image yielded better results than using the plugin directly. Enhanced images showed a contrast improvement with an average PSNR value increase of 40.73%, making the distinction between rock text and background noise more pronounced. Different color space transformations provided varying levels of enhancement, emphasizing the importance of selecting the appropriate transformation for optimal results. Moving forward, this research will continue to refine and enhance test outcomes. Future work will include comparing results from repeated experiments with those obtained using methods developed by other researchers.

Table 4.

Test outcome

8. Summary

Advanced image processing techniques can effectively restore cultural heritage images. For the Taikhar Rock cultural heritage texts, 650 images were used for training to distinguish genuine cultural texts from fake ones. This study applied the Mask R-CNN deep learning model to identify cultural heritage texts in rock inscriptions. The results highlight significant progress in using deep learning for archaeological research, particularly in the preservation and conservation of cultural heritage. By leveraging deep learning and computer vision, this method automates the identification and analysis of rock art, offering several key advantages:

• - Efficiency and Accuracy: Mask R-CNN enables more efficient and accurate detection and segmentation of rock art elements within large datasets. Automation reduces the time and effort required for manual inspection, allowing researchers to focus on advanced analysis and interpretation.
• - Systematic Documentation: Automating the detection process aids in the documentation, study, and preservation of ancient artworks for future generations.
• - Scalability: The Mask R-CNN model is highly scalable, making it suitable for analyzing diverse datasets with varying rock art examples. This is particularly valuable in Mongolia, which is rich in rock art sites.

Discovered rock art requires reconstruction through image processing techniques. Enhancing existing cultural heritage texts was a priority, and techniques such as noise reduction, CLAHE (Contrast Limited Adaptive Histogram Equalization), Gaussian filters, and enhanced local contrast filters proved effective. Additionally, color manipulation techniques provided greater control over the final image, addressing color balance issues, enhancing specific hues, and improving overall image quality.

Acknowledgments

I would like to thank G. Ankhsanaa, head of the Department of Conservation and Risk Research at the National Center for Cultural Heritage, Mongolia, for their collaboration and provision of research data.

References

• Adamopoulos, E. (2021). Learning-based classification of multispectral images for deterioration mapping of historic structures. Journal of Building Pathology and Rehabilitation, 6, 1-15. [https://doi.org/10.1007/s41024-021-00136-z]
• Angheluţă, L., & Chiroşca, A. (2020). Physical degradation detection on artwork surface polychromies using deep learning models. Romanian Reports in Physics, 72, 805. https://rrp.nipne.ro/2020/AN72805.pdf
• Bhat, M., & Patil, T. (2014). Adaptive clip limit for contrast limited adaptive histogram equalization (CLAHE) of medical images using least mean square algorithm. In 2014 IEEE International Conference on Advanced Communications, Control and Computing Technologies (pp. 1259-1263). IEEE. [https://doi.org/10.1109/ICACCCT.2014.7019300]
• Chaiyasarn, K., Khan, W., Ali, L., Sharma, M., Brackenbury, D., & DeJong, M. (2018). Crack detection in masonry structures using convolutional neural networks and support vector machines. In Proceedings of the ISARC: Proceedings of the International Symposium on Automation and Robotics in Construction (pp. 1-8). [https://doi.org/10.22260/ISARC2018/0016]
• Chen, R.-C., Dewi, C., Zhuang, Y.-C., & Chen, J.-K. (2023). Contrast limited adaptive histogram equalization for recognizing road marking at night based on YOLO models. IEEE Access, pp (99), 1-1. [https://doi.org/10.1109/ACCESS.2023.3309410]
• Ganbold, G. (2014). Detection of edges in color images. IEIE Transactions on Smart Processing and Computing, 3(6), 345-352. [https://doi.org/10.5573/IEIESPC.2014.3.6.345]
• Ganchimeg, G. (2013). Application exhibits of historical virtual museum. Proceedings of the International Conference on Electronics, Information, and Communication (ICEIC-2013), pp 188-189. Indonesia.
• Ganchimeg, G. (2015). History document image background noise and removal methods. International Journal of Knowledge Content Development & Technology, 5(2), 11-24. [https://doi.org/10.5865/IJKCT.2015.5.2.011]
• Gkioxari, G., Jalandoni, A., & Tuytelaars, T. (2021). Vandalism detection in rock art images using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). [https://doi.org/10.48550/arXiv.1703.06870]
• Hassan, E., El-Rashidy, N., & Talaa, F. M. (2022). Review: Mask R-CNN models. Nile Journal of Communication & Computer Science, 3(1), 1-10. https://njccs.journals.ekb.eg [https://doi.org/10.21608/njccs.2022.280047]
• Hatır, E., Korkanc, M., Schachner, A., & İnce, İ. (2021). The deep learning method applied to the detection and mapping of stone deterioration in open-air sanctuaries of the Hittite period in Anatolia. Journal of Cultural Heritage, 51, 37-49. [https://doi.org/10.1016/j.culher.2021.07.004]
• Hou, M., Huo, D., Yang, Y., Yang, S., & Chen, H. (2024). Using mask R-CNN to rapidly detect the gold foil shedding of stone cultural heritage in images. Journal of Heritage Science, 12, 46. [https://doi.org/10.1186/s40494-024-01158-9]
• Kwon, D., & Yu, J. (2019). Automatic damage detection of stone cultural property based on deep learning algorithm. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 42, 639-646. [https://doi.org/10.5194/isprs-archives-XLII-2-W15-639-2019]
• Le Quellec, J.-L., Duquesnoy, F., & Defrasne, C. (2015). Digital image enhancement with DStretch: Is complexity always necessary for efficiency? Journal of Digital Applications in Archaeology and Cultural Heritage, 2, 55-67. [https://doi.org/10.1016/j.daach.2015.01.003]
• Lobo, J. (2023). Deep learning-based graffiti detection: A study using images from the streets of Lisbon. MDPI Electronics, 13(4), 2249. [https://doi.org/10.339/electronics13042249]
• Meng, T., Huang, R., Lu, Y., Liu, H., Ren, J., Zhao, G., & Hu, W. (2021). Highly sensitive terahertz non-destructive testing technology for stone relics deterioration prediction using SVM-based machine learning models. Heritage Science, 9, 1-9. [https://doi.org/10.21203/rs.3.rs-98413/v2]
• Nazarian, E., Taylor, T., Weifeng, T., & Ansari, F. (2018). Machine-learning-based approach for post event assessment of damage in a turn-of-the-century building structure. Journal of Civil Structural Health Monitoring, 8, 237-251. [https://doi.org/10.1007/s13349-018-0275-6]
• NCCHM-National Center for Cultural Heritage of Mongolia. (2020). List of historical and cultural immovable monuments to be protected by Mongolia (№13). Retrieved from https://ncch.gov.mn/tangible
• Pizer, S. M., Amburn, E. P., Austin, J. D., Cromartie, R., Geselowitz, A., Greer, T., ter Haar Romeny, B., Zimmerman, J. B., & Zuiderveld, K. (1987). Adaptive histogram equalization and its variations. Computer Vision, Graphics, and Image Processing, 39(3), 355-368. [https://doi.org/10.1016/S0734-189X(87)80186-X]
• Sarkar, M., & Mandal, A. (2023). SLAAHE: Selective apex adaptive histogram equalization. Franklin Open, 3, 100023. [https://doi.org/10.1016/j.fraope.2023.100023]
• Thun, M. T., Sugiura, Y., & Shimamura, T. (2024). Poisson-Gaussian noise removal for low-dose CT images by integrating noisy image patch and impulse response of low-pass filter in CNN. Journal of Signal Processing, 28(2), 57-67. [https://doi.org/10.2299/jsp.28.57]
• USFM-Unified Statistics Foundation of Mongolia. (2023). Retrieved from https://1212.mn/
• Valero, E., Forster, A., Bosche, F., Hyslop, E., Wilson, L., & Turmel, A. (2019). Automated defect detection and classification in ashlar masonry walls using machine learning. Automation in Construction, 106, 102846. [https://doi.org/10.1016/J.AUTCON.2019.102846]
• Wang, N., Zhao, X., Zou, Z., Zhao, P., & Qi, F. (2020). Autonomous damage segmentation and measurement of glazed tiles in historic buildings via deep learning. Computer-Aided Civil and Infrastructure Engineering, 35, 277-291. [https://doi.org/10.1111/mice.12488]
• Zou, Z., Zhao, X., Zhao, P., Qi, F., & Wang, N. (2019). CNN-based statistics and location estimation of missing components in routine inspection of historic buildings. Journal of Cultural Heritage, 38, 221-230. [https://doi.org/10.1016/j.culher.2019.02.002]