# data[].attributes.attachment_link.description
Create a new spreadsheet and edit with others at the same time -- from your computer, phone or tablet. Get stuff done with or without an internet connection. Use Sheets to edit Excel files. Free from Google.

# data[].attributes.body
https://docs.google.com/document/d/1szGEf52pG63gGmz3Kn5oMkQyx8Y95wKi/edit?usp=sharing&ouid=101729400158839912982&rtpof=true&sd=true

# data[].attributes.body
https://www.scopus.com/record/display.uri?eid=2-s2.0-85128242002&origin=resultslist&sort=plf-f&src=s&st1=Road+Condition+Monitoring+Using+Smart+Sensing+and+Artificial+Intelligence&sid=d25b0661a878ce6f637a31576a05db90&sot=b&sdt=b&sl=88&s=TITLE-ABS-KEY%28Road+Condition+Monitoring+Using+Smart+Sensing+and+Artificial+Intelligence%29&relpos=0&citeCnt=19&searchTerm=

# data[].attributes.body
https://www.sciencedirect.com/science/article/abs/pii/S0950061819301813

# data[].attributes.attachment_link.title
Pavement structural evaluation based on roughness and surface distress survey using neural network model

# data[].attributes.attachment_link.description
Evaluation of pavement condition, which determines pavement maintenance and rehabilitation necessities, is inevitable using structural or non-structur...

# data[].attributes.body
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9697233/

# data[].attributes.attachment_link.readable_text
Lu Xiong, Academic Editor, Xin Xia, Academic Editor, Jiaqi Ma, Academic Editor, and Zhaojian Li, Academic Editor

Abstract

Road pavement condition assessment is essential for maintenance, asset management, and budgeting for pavement infrastructure. Countries allocate a substantial annual budget to maintain and improve local, regional, and national highways. Pavement condition is assessed by measuring several pavement characteristics such as roughness, surface skid resistance, pavement strength, deflection, and visual surface distresses. Visual inspection identifies and quantifies surface distresses, and the condition is assessed using standard rating scales. This paper critically analyzes the research trends in the academic literature, professional practices and current commercial solutions for surface condition ratings by civil authorities. We observe that various surface condition rating systems exist, and each uses its own defined subset of pavement characteristics to evaluate pavement conditions. It is noted that automated visual sensing systems using intelligent algorithms can help reduce the cost and time required for assessing the condition of pavement infrastructure, especially for local and regional road networks. However, environmental factors, pavement types, and image collection devices are significant in this domain and lead to challenging variations. Commercial solutions for automatic pavement assessment with certain limitations exist. The topic is also a focus of academic research. More recently, academic research has pivoted toward deep learning, given that image data is now available in some form. However, research to automate pavement distress assessment often focuses on the regional pavement condition assessment standard that a country or state follows. We observe that the criteria a region adopts to make the evaluation depends on factors such as pavement construction type, type of road network in the area, flow and traffic, environmental conditions, and region's economic situation. We summarized a list of publicly available datasets for distress detection and pavement condition assessment. We listed approaches focusing on crack segmentation and methods concentrating on distress detection and identification using object detection and classification. We segregated the recent academic literature in terms of the camera's view and the dataset used, the year and country in which the work was published, the F1 score, and the architecture type. It is observed that the literature tends to focus more on distress identification ("presence/absence" detection) but less on distress quantification, which is essential for developing approaches for automated pavement rating.

Keywords: deep learning, image segmentation, pavement surface condition index

1. Introduction

Two vital elements of road pavement (referred to as pavements in the rest of this paper) management are inventory management and periodic condition evaluation; both are used to set future priorities for pavement construction management and maintenance. In this paper, pavement refers to hard surfaces used for motor vehicles. A complete pavement management system consists of inventory data collection (i.e., width, length, shoulder, and pavement type) and pavement characteristic assessment, i.e., (roughness (ride), surface condition (distresses), surface skid resistance, pavement strength, and deflection). The current pavement networks, including motorways across a country, are developed and modernized over centuries. The construction, width, and length of a pavement depend on the traffic it will carry and the type of connection it will make. They are classified into different categories; for example, in Ireland, they are classified as motorways, national primary, national secondary, regional roads, and local roads [ 1]. A common way to periodically evaluate surface condition, including distresses on a pavement network, is for the civil authority to conduct a visual surface condition assessment and a ride smoothness test. Surface condition is assessed through visual surveying and usually consists of three steps: (1) pavement condition data collection, (2) distress identification and quantification, and (3) assigning a pavement rating index to a stretch of a pavement using a standard rating scale (e.g., pavement surface evaluation rating-PASER [ 2]) that is typically localized to a specific geographical region [ 3]. Figure 1 gives a complete picture of the three-step process. The data collection is followed by distress occurrence, severity measurement, and pavement condition rating decisions.

Data collection, the first step of surface visual assessment, is usually carried out by specially adapted vehicles (or, more recently, on devices such as smartphones [ 4] or unmanned aerial vehicles) for visual surface surveying. The vehicle is fitted with a computer, Global Positioning System (GPS) sensor, and an imaging sensor. In step 2, pavement distresses are identified and quantified using their shape, size, and texture. Due to environmental and geographical conditions and the actual pavement construction process, pavement distresses may vary in shape, size, and texture. Variations can also be caused by different image capture technologies and the placement of sensors in specialized vehicles used to collect pavement data. In step 3, a rating is assigned to a stretch of pavement based on distress identification and quantification from step 2. A rating is applied to an initial stretch after inspection and then will be adjusted along the road if the pavement surface changes noticeably. The length of the stretch of road typically ranges from 50 m to 200 m, while the width of the stretch ranges from 4 m to the entire width of the road. The rating is performed directly by civil authority staff or subcontracted to private companies. Civil authorities use this condition rating to estimate pavement service life and treatment measures to improve the condition.

Maintenance and improvement of pavements are expensive. For example, Ireland's government spent 850 Million Euros in 2021 to improve and maintain local, regional, and national primary and secondary roads [ 5]. There are 5413 km of national highways (primary, secondary, and motorways), 13,124 km of regional roads, and 81,300 km of local roads in Ireland. It totaled 99,830 Km of road network in 2018 in Ireland, meaning 95% of the road network in Ireland consists of regional and local roads [ 5]. Moreover, it takes most of the year to complete mechanical surveys on the national highways which are only 5% of the network, therefore we need a quicker method for the other 95%. Manual rating requires cognitive skills built through extensive training and experience. It is also impossible for a manual rater to transverse the whole road network across the country in a specific time. It is a challenging process and prone to errors. To make the process faster, more economical, and reliable, researchers have investigated automated processes for pavement condition evaluation, usually based on computer vision, machine learning, and, more recently, deep learning [ 6, 7, 8, 9]. In recent years, researchers have reviewed different data acquisition technologies, including 1D-sensors, 2D-sensors, and 3D sensors, to automate pavement conditions [ 10, 11, 12, 13, 14, 15]. Commercial solutions for automatic pavement assessment with certain limitations exist; the topic is also a focus of academic research. More recently, academic research has pivoted toward deep learning, given that image data is now available in some form. However, research to automate pavement distress assessment often focuses on the regional pavement condition assessment standards the country or state follows.

This paper contributes a list of significant pavement condition rating indices (segregated based on granularity and measurement criteria) used in various parts of the world. A comprehensive list of distress for asphalt and concrete roads is presented and segregated into six main groups. Commercial solutions for data capture and assisted image analysis are reported along with their limitations. We then present a comprehensive list of publically available datasets along with a link to download, which is segregated based on view type, type of distress, resolution, type of ground truth, number of images available, and country of origin. The review of recent (2018-2022) deep learning techniques for pavement distress detection, classification, segmentation, and direct pavement rating classification is presented. We segregated the literature in terms of the camera's view and the dataset used, the year and country in which the work was published, the F1 score, and the architecture type that helps identify the latest trends. We observe that much of the literature focuses on automating step 2-distress identification and quantification- while there is less emphasis on automating step 3-automatically computing a pavement rating. We observe that the criteria a region adopt to make the evaluation qualitative depend on factors such as pavement construction type, type of road network in the area, flow and traffic, environmental conditions, and region's economic situation.

This paper is organized as follows. Section 2 explains the type of pavement surfaces, the types of distresses, and pavement rating indicators used around the world, including their advantages and limitations. Section 3 reviews data collection techniques for visual pavement inspection and commercial practices. Section 4 generalizes an automated rating system and publically available dataset and reviews classical and deep machine learning approaches. Then, we discuss the limitations of an AI-based automated pavement rating system. Finally, we conclude in Section 5.

2. Pavement Surface Types and Distress Assessment Indicators

This section briefly explains various pavement types, visual pavement distresses, and pavement assessment indicators.

2.1. Pavement Surface and Distress

Pavement or road surfaces can be categorized into four general classes, i.e., asphalt, concrete, gravel, and brick and block [ 16]. Asphalt, also known as flexible pavement, is widely used to construct national, regional, or local roads across the road network and has different sub-categories depending on its construction. Over 90% of the total European road network has an asphalt surface. Concrete surfaces are usually used in urban environments and can be subdivided into joined cement concrete and continuously reinforced concrete surfaces [ 17]. Concrete pavements are expensive and time-consuming to construct, but they are typically more potent and durable than asphalt roadways. They are more common in the USA; for example, approximately 60 percent of the interstate system in the USA is concrete. Pavement condition assessment considers several pavement characteristics, i.e., roughness, surface condition (distress detection), surface skid resistance, and pavement strength. Surface condition plays a significant role in pavement assessment, which requires pavement distress detection and quantification. Pavement surface distresses that occur in different geographical regions can be divided into six groups, i.e., cracks, surface openings, surface deformation, surface defects, joint deficiencies, and miscellaneous distress [ 17, 18] (see Table 1 ).

Table 1

Surface Distress Group

Asphalt Rural Flexible

Asphalt Urban Flexible

Joined Portland Concrete

Continuously Concrete Reinforced

Block cracking

Reflection cracking at joints

Reflection cracking at joints

Depression and bumps

Shoving, depressions, bumps, sags, and heave

Spalling of longitudinal and transverse joints

Spalling of longitudinal and transverse joints

Most of these distresses can be detected generally through visual inspection (standard practice) of pavement surfaces, and their severity and quantity can be recorded using manual measurement tools [ 17]. Visual distresses appears on the surface due to wear and tear, which may indicate a fault in the construction. It may appear differently in rural and urban regions, depending on the surface type, the severity (low, medium, high) of the underlying problem, and other environmental conditions.

2.2. Pavement Assessment Indicators

Measuring different pavement characteristics is essential in long-term pavement performance incorporating all or a subset of pavement characteristics to conduct pavement assessments. These condition rating systems vary from country to country (or within a state in the USA), considering local variations, the characteristics of the pavements, and economic conditions.

Pavement characteristics that are generally separately measured include pavement roughness, a vital pavement characteristic measured on a rating index known as the International Roughness Index (IRI) [ 19]. It is estimated in a moving vehicle from a longitudinal pavement profile with sensors capable of measuring vertical movement [ 20, 21]. Another essential characteristic is transverse deflection, also known as rut depth, measured manually or using sensors that generate transverse pavement profiles [ 9]. Visual pavement condition assessment requires distress detection and quantification to measure pavement conditions and is more reliable than other methods are. Engineers and professionals have proposed several standards for visual surface assessment, such as Pavement Surface Evaluation Rating (PASER) [ 2], Pavement Condition Index (PCI) [ 19, 22], Pavement Surface Condition Index (PSCI) [ 23], and the Road Condition Indicator (RCI) [ 24]. Table 2 lists different pavement condition ratings used around the world. The standard ratings of various regions differ in scale granularity, formula to estimate a value on the rating scale, and data acquisition procedure.

Table 2

Type of Indicators

Granularity

Measurement Criteria

Standard Developing Body

Integer value

Pavement Condition Rating (PCR)

Alabama Department of Transport

Pavement Structural Condition (PSC)

Washington Department of Transport

Surface Condition Rating (SCR)

Georgia Department of Transport

Integer value

Integer value

Repair Requirement Index (RRI)

0-5 New - More than 12 (Lifetime over)

A mathematical formula based on International Roughness Index, crack rate coefficient, and pothole rank coefficient

Japan (after 2005)
Tajikistan

The earliest work in creating a standardized condition assessment scale dates from the 1960s in the United States [ 25]. The scale used two pavement characteristics-pavement roughness and visual surface distress identification, to determine the Present Serviceability Index (PSI) ranges from zero (very poor) to five (very good condition). A roughness index was carried out by 3-5 individual raters trained to qualitatively estimate pavement roughness by driving a vehicle on the pavement. It was followed by visual inspection for cracks, patches, and potholes. These two were then combined mathematically to calculate the PSI score (0-5) [ 25].

Over the years, data acquisition techniques have evolved; different pavement condition assessment ratings have been proposed that mainly focus on assessing the different types of pavement characteristics, their quantity, and their effect on the overall condition of the pavement. PASER is a direct rating on a scale of 10-1 (9-10 is excellent condition, while 2-1 is extremely poor). On the other hand, the ASTM standard for pavement is PCI, a rating on a scale of 100-0 (85-100 is a good condition, while 0-10 is completely deteriorated. It is mathematically based on distress occurrence and severity level. The Irish PSCI [ 18, 23, 26] rating is on a scale from 1-10, similar to PASER, where index-1 is the lowest (surface completely worn out or failed), and index-10 (no distress, new pavement) is the highest. It covers flexible urban pavements, urban concrete pavements, and flexible rural pavement separately. PSCI ratings are given to continuous stretches of pavements with similar conditions, with 200 m being the minimum length to have their distinct rating [ 26]. In the United States, the Federal Land Transportation program recommends visual distress detection based on PASER for direct pavement condition evaluation [ 27]. Some transportation departments (or road authorities) that use scales similar to PCI use a subset of the visual distresses and roughness index to calculate the PCI rating. For example, the New Zealand Road Assessment and Maintenance Management System (RAMM) assigns a CI (Condition Index) from 0-100 (0-Excellent-100-Failed); it includes a visual inspection of not only the pavement but the surface water channels along the pavement [ 28]. China uses the Chinese Pavement Condition Index (CPCI), a scale similar to PCI, and considers cracking, raveling, potholes, rutting, and roughness. Japan used the Maintenance Control Index (MCI) until 2005, a function of cracking, rutting, and roughness, on a scale of 10 to 0 [ 29]. After 2005, the Ministry of Transportation Japan has used RRI, which is a function of cracking ratio, rutting depth, and International Roughtness Index [ 29]. A similar index is used in Tajikistan under Japan International Cooperation Agency [ 30].

The RCI is a rating from 1-4 (with 1 meaning no physical deterioration, while 4 is severe deterioration), adopted in England, Wales, Scotland, and Northern Ireland, and fuses visual condition and gauging parameters of pavement condition [ 24]. In Germany, the RMA (Road Monitoring and Assessment) protocol rate the pavement into four categories based on visual distresses [ 31]. Some states use four classes in the USA, i.e., Good, Fair, Poor, Very-Poor, as a condition scale based on the original PSI rating. In some countries, such as India and Brazil, a visible pavement distress condition rating on a scale of 0 to 3 is used [ 32]. Ratings are based on cracking, rutting, raveling, patching, and potholes, while roughness is not considered [ 3]. Pavement condition surveys of national and local roads are commonly conducted annually, every two years, or every five years in different regions across the world (for example, in Ireland, they are conducted every two years, while in Florida, state highway surveys are completed annually [ 33]). Therefore, these survey methods should be quick, fast, reliable, and economical.

In summary, different regions have different ways of performing pavement condition rating; some take roughness and visual condition combined to assign a rating from a standard scale (e.g., China, Japan, and some states in the USA), while others rate only a subset of visual distress (e.g., UK, Ireland, Brazil, Germany, New Zealand, India, and some states in the USA). Some of these indices are very granular (1 to 100) such as PCI in some parts of USA versus that (0 to 3 scale) used in Brazil/India. The choice of scale has evolved with economic prosperity and maturity of the road network.

3. Data Acquisition Process and Commercial Practices

This section describes how the data is acquired for pavement distress assessment and current commercial practices.

3.1. Data Acquisition Process

Different sensing technologies, sensing positions, and vehicles have been used to capture data to assess pavement conditions. The choice of technology depends on economic factors, availability of resources, and pavement characteristics to be measured [ 4, 34]; the sensor's position depends on the sensing technology used to acquire data for pavement condition assessment [ 18, 26]. Figure 2 lists three types of sensing technologies available for structural monitoring and distress detection that can be integrated into vehicles with a GPS.

To measure the pavement surface's vibration, deflection, displacement, stress, temperature, or humidity, 1D or point sensors are usually used for structural condition assessment, which is an indirect method of pavement surface condition assessment. Data from 2D or 3D sensors is generally used for visual distresses identification and direct pavement condition assessment. 3D sensors, including laser imaging, stereo pair, and ground penetrating radar, are used obtained from the top view of pavement. In contrast, 2D sensors, including RGB (color) cameras, are used mainly in the frontal wide-view camera configuration.

Recently, research in [ 35, 36, 37, 38, 39, 40, 41, 42] has shown promise in using aerial vehicles to help detect visual distress, such as potholes, cracks, and aging on asphalt pavement. Data collection using aerial imagery poses other difficulties, such as occlusion due to ongoing traffic, permission to fly in urban areas, and lower ground sampling distances. However, it does have limited use in pavement condition assessment, especially on airport runways [ 41]. Using a laser or color camera mounted at the back of the vehicles, a top view of the road, as used by [ 43, 44, 45, 46, 47], focuses on crack detections and potholes. A wide-angle view of the road, using a camera mounted on the front of the dashboard or top of the car, as used by [ 32, 48, 49], is used for detecting types of cracks, potholes, and types of surfaces and surface ratings.

Hand-held mobile cameras, as used by [ 47, 50, 51, 52, 53] have significant utilization in road surface distress detection. The top-view camera setup provides a better ground sampling distance than the wide-view setup, while the wide-view is much quicker as it covers more area per image. Thus, the literature review highlights that different camera capturing techniques for visual distress detection have been used: frontal wide-view, top-view, hand-held smartphone, and aerial view.

3.2. Current Commercial Practices

Many commercial systems are available for image data collection for pavement condition assessment. These systems are reconfigurable and can be customized to carry different data sensors and inbuilt data analysis software for manual or semi-automated rating. The inbuilt software uses automated image analysis techniques to detect and quantify visual cracks for a pavement rating system. This section discusses currently available reconfigurable commercial systems used in different regions for data collection and assessment. A commercial vehicle usually consists of a GPS/GNSS module, transverse profile logger for rutting, laser profilometer for roughness estimation, high-resolution odometer, laser cracking measurement system, video logging modules for frontal wide-view capture, bump integrator, and an onboard computer for recording data (see Figure 3 ).

PaveVision3D [ 12] is a system that contains a data vehicle, an automated surface imaging system capable of conducting a complete lane width distress-detection survey at 1-mm resolution at a speed up to 100 KM/h. It uses a top-view approach with laser scanners and an intensity camera looking down on the pavement. It has dedicated software for crack identification, optional software and hardware for laser rut measurement, and laser roughness measurement. Pavemetrics [ 33] provides a similar solution called Laser Crack Measurement System (LCMS), which uses 3D laser scanners fitted on a vehicle. The LCMS software can geo-tag, measure, detect and quantify cracks, potholes, bleeding, shoving, raveling, and roughness. It can capture one lane of the pavement with a 1-mm resolution and a speed of 100 KM/h. The automated IRI and distress detection reports produced by LSTM comply with ASTM and the American Association of State Highway and Transportation Officials (AASHTO). These specialized vehicles are costly, and the distress detection software is calibrated for national highway pavement conditions in the USA or Canada.

In England, Wales, Scotland, and Northern Ireland, the pavement maintenance authorities use TRACS (Traffic-speed Condition Surveys) and SCANNER (Surface Condition Assessment for the National Network of Roads), which consists mainly of a laser scanner mounted on the front and back of a van giving a top view of the pavement surface. ROMDAS [ 54] is another customizable data-capturing solution that can provide both top-view using laser scanners for crack measurement or frontal view using the color camera for other detecting other distresses. STIER [ 55] is a customizable vehicle with a top-down stereo vision monochrome camera and a frontal view camera for data capture. It uses its software to detect distresses defined by German FGSV regulation, i.e., cracks, potholes, inlaid patches, applied patches, open joints, and bleeding. This information is then used to rate roads into four categories [ 31].

PMS video survey van [ 56] is equipped with distance-measuring sensors and a GPS sensor attached to the onboard computer to provide accurate distance measurements. A frontal wide-view video camera mounted on the dashboard of the pavement surface offers a high-quality compressed video stream using a state-of-the-art compression algorithm to retrain high definition (1920 × 1080) at minimum storage space. The real-time software integrated with the onboard computer provides options for the expert to rate the pavement condition on the go or record the video for offline manual PSCI rating. The ground sampling distance is lower than cameras providing top-view imaging. However, it can cover the whole pavement in one direction on multi-lane pavements with imaging every 5 m.

The speed of data collection through the PaveVision3D, Pavemetrics, and Ricoh (with a top-view 3D laser camera) [ 57] compared with less expensive dedicated vehicles [ 56] (with a high-resolution frontal wide-view camera) is comparable. However, the wide-view camera systems can cover more lanes than the top-view systems due to frontal coverage. The amount of data generated for records and further computation using top-view vehicles is more than from vehicles that use only frontal view for paving rating. The methods using 3D laser sensors (PaveVision3D, Pavemetrics, or RICOH) produce better ground sampling distance per pixel than frontal wide-view images. Still, the confidence in the final output of such reliable vehicles is much more than using a GoPro camera or a smartphone, as they do not have a customized processing unit to fuse readings from different sensors such as GPS and distance measuring sensors.

Therefore, a vehicle with a wide-view camera in front of a dashboard without external sensors (laser scanners or profilers), is more economical for an extensive network of local and regional roads with less maintenance requirement. It requires less storage to record pavement images by compromising spatial resolution; however, enough distress information to manually rate a pavement condition on a standard scale. On the other hand, vehicles with a top-view camera and external sensors are recommended for national highways and motorways [ 2]. They provide a higher spatial resolution, better for distinguishing different types of cracks and patches. Such vehicles have higher maintenance costs and would not be cost-effective when driven on the regional or local road network.

The commercial solutions discussed are usually limited to automated data capture and semi-automated analysis for distress identification and quantification, followed by an assisted or manual pavement condition rating assessment for a stretch of pavement. Some companies in the USA and Japan do provide automated solutions for pavement condition ratings. RoadBotics [ 58] working locally on USA roads, use a limited version of PASER [ 2], i.e., they rate pavement from 1 to 5, with 5 being the lowest rating. An automated rating system from Ricoh [ 57] estimated the amount and location of cracks on a 50 cm × 50 cm patch and has adopted its rating system for Japanese roads based on PCI. The automated solutions for frontal wide-view and top-view are still evolving toward robustness and generalization and require calibration and transfer learning with local data.

In summary, there is no off-the-shelf solution for automated pavement condition rating. Most of the existing commercial solutions usually provide automatic data and image-capturing solutions, while their ability to detect and quantify distress from images is limited to a few distresses. The limited automated solutions for intelligent distress detection (identification and quantification) from imagery require recalibration to capture regional variations in the pavement distresses for shape, size, or texture due to environmental conditions. These automated solutions also do not support adaptation to different pavement condition rating standards used by different regional and local authorities. The choice of imaging technology for visual inspection depends on the type of distress, environmental conditions, and economic factors and how adequate they are in identifying those distresses.

4. Literature Review on Automated Visual Pavement Condition Rating Systems

Automated visual pavement surface condition rating can be broken down into several processes: a pavement surface classification process, a distress detection and quantification process, prediction of a rating score computed using the type of distress detection and its quantification based on a standard rating scheme, and predicting the rating for a given stretch of pavement based on majority voting scheme. The manual PCI system works similarly, i.e., it identifies all the individual distresses and quantities, calculates 'deduct' values based on each distress type, severity, and amount, and then generates an overall rating by subtracting the sum of the weighted deduct values from a perfect score of 100.

To generalize the above statement, let D 0 to be the 0 to nth distinct types of distresses, be the area of the mth instance of the nth distress, and is the effect or weight on the rating score of the nth distinct distress, then we can define the rating score of a pavement condition in an image using Equation (1).

S c o r e 10−1 =n o r m a l i z e 1−10(∑ o m∑ o n w n A m D n)

where is the distress, and n is the type of distress. is the area in m_th instance of and is the weight of each n_th distress to overall score.

For decades, extracting useful information from images has been a task of computer vision-based systems. Early researchers used image processing techniques (such as gradient or change in intensity detection, color or intensity thresholding, and morphological processing) to extract useful information from the pixels [ 59] directly. We first present a few prominent image analysis techniques and their limitations. Then we present a brief history of the evolution of machine learning techniques, benchmarking, and state-of-the-art models deep learning models for pavement condition assessment.

4.1. Evolution of Machine Learning in Computer Vision

With the development of machine learning algorithms such as K-mean classifiers [ 60], support vector machines (SVM) [ 61], ANN (artificial neural networks) [ 62], and many others [ 63], researchers started using hand-crafted features such as SIFT (scale-invariant feature transform) [ 64], ORB (Oriented FAST and rotated BRIEF) [ 65], or AKAZE [ 66] to uniquely describe an image, object, or region of interest. Image processing algorithms use these features to learn to classify, detect uniquely, or segment objects, areas of interest, or images. Hand-crafted features and classical machine learning provide robustness across scale, lighting, rotation, and other environmental conditions. Advancements in machine learning, including the development of dense neural networks [ 67], convolutional neural networks (CNN) [ 68], and more recently, Transformers [ 69], has provided solutions for computer vision tasks, which are more robust to changes in the input data and are coined as 'deep learning' computer vision or image analysis techniques. Handcrafted features are automatically extracted for a particular computer vision problem using deep learning algorithms.

4.2. Automated Distress Detection and Identification

The review of the literature tells us that researchers have investigated pavement distress detection using different imaging technologies, computing suitable features, and learning data models to detect, classify, or segment distresses over the last decade. In [ 7], the authors listed technologies to enable researchers to choose the imaging technique for pavement stress detection. They mentioned the state-of-the-art methods using image processing techniques for crack detection and potholes detection while highlighting the problems that need to be investigated, including pavement texture detection, temperature segregation detection, rutting detection, and joint faulting detection. The distress identification literature can be segregated into image processing, classical machine learning, and deep learning techniques.

4.2.1. Publicly Available Datasets

Over the years, researchers have made available datasets for benchmarking automated distress detection systems, mainly covering different types of cracking and potholes [ 4]. Only a few focus on other distresses or visual pavement classification. In [ 14], the authors have reviewed different methods to detect pavement surfaces and highlighted different benchmarks for pavement surface detection. In [ 4], the authors list contributions to existing publicly available pavement image datasets for distress detection. These very limited datasets can be categorized based on the view angles (top-view, wide-view, hand-held), and imaging technologies (3D or intensity), mainly focused only on a subset of distress types (different crack types, potholes, and patches) found locally in the geographical regions (USA, China, India, Japan, Czech Republic, Brazil, Italy, and Mexico). In [ 70, 71], the authors generated a pavement distress detection dataset using images available from Google APIs. The images available through Google APIs give both top-view and wide-view images; however, the images are old, captured over the years, and not labeled for pavement distress. The authors in [ 4] highlight that most of the literature on distress detection is based on image datasets not publicly available. Table 3 summarizes the current publically available datasets for distress detection and pavement condition assessments. The datasets are used as benchmarks to verify the crack segmentation algorithms include CrackTree200 [ 72], Crack500 [ 73], CrackForest [ 74], and Agile-RN [ 70]. Though several researchers have used it for verification of their deep learning-based architectures; however, they are limited in terms of covering various shapes, sizes, and textures of cracks formed due to different environmental conditions.

Table 3

S.No

Name

Distress

Ground Truth

Device

No. of Images

Resolution

Ch

View

Country

Link

3

CRACK500 and CRACK-500-B

cracks

pixel-level

hand-held static

500 + 1896

2560 × 1440
640 × 360

3

Top

China

https://bit.ly/3QPeAsx

8

RDD2020 (excluding Japan)

pothole, longitudinal crack, transverse crack, alligator crack

bounding box

mobile device moving vehicle

11,000

720 × 720 (India) 600 × 600 (Czech)

3

Frontal

India Czech Republic

9

RDD2022 (excluding RDD2020)

pothole, longitudinal crack, transverse crack, alligator crack

bounding box

mobile device moving vehicle

17,500

3650 × 2044 (Norway) 640 × 640 (USA) 512 × 512 (China)

3

frontal/top

Norway USA China

GAPS (German Asphalt Pavement Distress Dataset) used by [ 31, 71] provides a top-view, good quality, close range, high-resolution dataset (approx ~2468 images) for surface distress identification which trained operators to label in the field. Six different distress defined by German FGSV regulation [ 71], i.e., cracks, potholes, inlaid patches, applied patches, open joints, and bleeding, are labeled in the images using a bounding box. The dataset is limited to only a few distresses regulated by German FGSV and does not contain severity levels of these distresses.

The second main contribution to the distress detection dataset is by [ 75], which has three different variants, namely, RDD2019, RDD2020 [ 75], and RDD2022 [ 76]. The dataset contains frontal-view images that are mainly labeled using a bounding box for four distresses, i.e., alligator, transverse and longitudinal cracks and potholes. The 2019 variant contains images of Japan, while the 2020 variant contains images from India and the Czech Republic. The 2022 variant contains images from China, Norway, and the USA. The dataset may be prone to labeling errors as it is labeled using crowdsourcing by labelers, not an expert in the field. A similar dataset for cracks with a wide-view camera located at the back of the vehicle is contributed by [ 77].

Two frontal view datasets focus on pavement rating; the first is the Paris-Saclay and the second is the Road Quality Dataset (RQ) [ 78]. The Paris-Saclay dataset [ 79] is annotated for pavement condition rating for a stretch of a road based on PASER for New York roads. The frontal-view images are extracted from Google Maps API, while the ground truth annotation for each stretch is extracted from the pavement condition rating of New York in [ 80]. The ground truth annotation contains the street index, the number of images in the street, the PASER rating for each street segment, and a rating of Good, Fair, and Poor for each street segment. A similar image dataset can be extracted from Google images for Oakland, USA, while the street segment pavement rating based on PCI can be generated from the database available at [ 81].

RQ Dataset [ 78] is a manually annotated frontal-view image for pavement condition index ratings based on six different condition ratings for the Czech Republic. The pavement condition rating criteria are defined in [ 78], while the images are obtained using Google Maps API. FHWA-LTPP [ 34] is another image-level classification resource composed of for five distress (alligator, longitudinal and transverse cracking, deflection, and longitudinal profiles) captured from different states of the USA.

4.2.2. Image Processing Techniques

Techniques using decision-based rules and image processing mainly focus on crack segmentation and identification. The authors in [ 12] described different image processing techniques for edge detection to find surface defects and segregated the recent literature on pavement stress identification using machine learning models into classification, object detection, and pixel-level segmentation problems. The authors of [ 82] proposed a modified Otsu-Canny edge detection algorithm for pavement crack detection. They evaluated the technique on a publicly available dataset Crackforest [ 83]. Peng et al. [ 84] proposed a double thresholding segmentation technique. After applying an enhanced Otsu threshold segmentation algorithm to eliminate pavement symbols in a runway image, they applied an adaptive iterative threshold segmentation algorithm. Lastly, the shape of the crack is achieved through the morphological denoising technique. In [ 85], the authors propose a multiscale local optimal threshold segmentation for pavement crack segmentation and crack density distribution. The method achieves better results than the optical threshold and global thresholding techniques. Zhao et al. [ 86] proposed an improved pavement edge detection method for crack identification. In [ 87], authors have used image processing, including thresholding, filtering, and morphological processing, to identify fatigue cracks. CrackIT [ 88] uses image pre-processing techniques before applying machine learning models for crack detection.

Image processing techniques are mainly applied to pictures with a top view of the pavement. Moreover, the early literature focuses on identifying characteristics of cracks or potholes. The image processing techniques are less robust to changes in intensity, noise, environmental factors, and pavement construction variations.

4.2.3. Classical Machine Learning Techniques

Machine learning approaches for distress identification can be classified as an image or object classification problems, object localization or detection problems, or pixel-segmentation problems. Many classical machine-learning approaches have been investigated for crack detection, including [ 89, 90, 91]. Daniel et al. [ 92] proposed a method to detect and classify cracks and potholes on asphalt pavements. They offered a two-step approach, pavement defect detection and classification, and defect severity detection and evaluation. The second stage is important for an automated pavement condition assessment and computed defect severity for each defect by calculating the area of the blobs. The method achieved 86% classification accuracy for cracks and potholes.

Raveling is a common visual distress in asphalt pavements, which occurs due to the loss of surface stones. It is recognized visually by observing the change in the macrotexture of the asphalt pavement along the stretch of the pavement. The severity of raveling increases with a higher chip loss from the pavement surface. In [ 93], authors evaluated different classical machine learning techniques such as AdaBoost with decision trees, support vector machine, and random forest to detect and classify different levels of raveling severity. For data collection for raveling, they used 3D images from PaveVision3D [ 12]. They observe that random forest is better than other techniques, with a recall ranging from 86.9% for level -1 severity to 75.6% for level-3 severityVery little work is reported on raveling detection and severity classification. In [ 94], the authors highlight the limitations in generalizing classical machine learning methods for crack detection. In [ 4], the authors have listed many classical machine-learning approaches for distress detection. These approaches mainly focus on detecting fatigue cracks, longitudinal and transverse cracks, potholes, rutting, and raveling. The image dataset is mainly captured through the top-view camera on a specialized vehicle, a hand-held camera view, or a UAV. Different hand-craft features have been extracted in these techniques. Models such as K-nearest neighbor, support vector machine, artificial neural network, and random forests are used to train a pixel-classifier (image segmentation) or an object detector. The precision ranged from 65.8% to 99% and recall from 79.4% to 98% [ 4].

However, these evaluation parameters are not generalizable as they depend highly on the image capture process. The datasets used mainly contain localized cracks or potholes, do not have different severity levels and are limited to a particular pavement type in a specific geographic region.

In summary, the public datasets available are limited to certain distress and mainly annotated by presence or absence of the distress. The image dataset annotated for pavement rating indices is also limited and does not cover the full range of standard visual rating scales, i.e., PASER [ 2] and PSCI [ 18]. The dataset does not cover distinct types of distress (mentioned in Table 1 ), or different shapes and textures, which vary due to different viewing angles, camera sensors, and geographical locations. Therefore, the evaluation matrix based on these benchmarks is less helpful in developing real-world automated pavement condition assessment systems.

4.2.4. Deep Learning Techniques

We focused on literature from 2018 onwards for deep learning techniques. The techniques are mainly broken into segmentation, classification, and object detection algorithms. Deep learning techniques, mainly convolutional or filtering layers, require a large amounts of data. Deep learning techniques are now widely used for computer vision tasks, including semantic segmentation, image classification, object detection, and image generation [ 95]. Deep architectures have also been used to solve classification hyperspectral imagery for remote sensing [ 96, 97].

Deep learning algorithms or architectures mainly consist of two parts the feature extraction phase and a classification, segmentation, or detection phase. In simple terms, for a deep learning-based classification, the CNN provides feature extraction layers, and the dense neural layer is added to estimate a class based on a feature extracted by the CNN. In deep learning-based segmentation, the CNN provides a feature extraction layer and is termed an encoder, while a set of de-convolutional layers are added to obtain pixel-level classification and termed a decoder layer. In deep learning-based object detection or localization, the CNN is used for feature extraction, followed by region proposal layers for object detection bounding box on the original image [ 68]. The interlinked deep learning layers are usually termed as 'architectures' or, when referred to alongside the weights and biases, as models.

Distress detection using a deep neural network can be separated into object detection, segmentation, and classification-based approaches [ 4, 8]. One major bottleneck for developing a model using deep learning is a good set of balanced training data for different distresses in the images, instances, and quantity [ 98]. In [ 99], authors present the first CNN-based raveling detection by training macro texture features obtained from the 3D images from PaveVision3D [ 12]. They achieved the highest accuracy of 90.8% for different raveling detection and an 85% accuracy for severity classification.

Classification Approaches to Distress Detection

Classification-based distress detection focuses on whether an image or part of the image is classified as a particular type of distress. The authors in [ 100, 101] proposed a flexible pavement distress classification convolutional neural network (CNN) framework to classify whether a patch is a crack or not. The images used are taken from a hand-held mobile phone camera. They evaluated the accuracy of their approach by comparing it with different classification approaches. Aparna et al. [ 102] assessed the feasibility of hand-held thermal imaging for pothole patch classification. Image data is acquired under various lighting conditions with offline data augmentation. A residual CNN model with pre-trained weights gave an accuracy of 99.7% for pothole patch image classification with an image size of 224 × 224 pixels. Yusof et al. [ 103] proposed a multi-label classifier for crack-type classification, i.e., transverse, alligator, and longitudinal. The images were taken from a hand-held Nikon digital camera with a dimension of 1024 × 768 pixels. The image was broken into a 32 x 32 patch image to classify different crack types. The data collection was carried out for Malaysian pavements.

An average accuracy of 98% was achieved to classify crack types with a precision of 97%. In [ 104], authors presented an algorithm for occurrence and severity classification in images captured from a top-view camera of urban flexible pavements in Spain. Their occurrence detection is based on patch classification using ResNet architecture, while the severity classifier is also a ResNet architecture. Each image is cropped to remove the background, broken down into three smaller blocks, resized to 224 × 224 pixels, and labeled for six classes, i.e., alligator cracks, longitudinal cracks, transverse cracks, pothole, raveling, and patches. To determine the severity of four distresses, mainly longitudinal cracks, transverse cracks, potholes, and patches, they labeled each distress with a bounding box in each image block. Although there were multiple distresses in each image block, the smaller block size minimized the likelihood of having different types of distress in each block. For the distress occurrence stage, the classifier's average F1-Score was reported to be 0.9262 on validation data, while the average Intersection of Union (IoU) was 0.729.

Researchers [ 31, 105, 106, 107, 108, 109] have also used a similar patch-based approach, i.e., dividing a higher resolution image into small image patches to detect localized distress, i.e., distinct types of cracks and potholes. In summary, most classification-based approaches focus on identifying types of distress in an image patch of higher-resolution images. Localized distresses are investigated, i.e., potholes and cracks. The images are taken from a top view or a hand-held camera view; the data set is localized to only specific to one region. The number of image patches is reasonable in number with a limited higher resolution image from where the patches have been extracted.

Many researchers have used patch or image classification techniques for multiple distress detection. Researchers in [ 43] and [ 44] used top-view color images for the experiment and then used ResNet-based architectures to develop a model for multiple distress classification. The ResNet model used in [ 43] has an F1-score of 0.92, whereas the model used in [ 44] has an F1-score of 0.90 on their test datasets. An image classification technique for multiple distress detection is mainly used for bleeding, raveling detection, and severity classification by [ 110] for pavements in Iran and [ 99] for pavements in the USA. In [ 46], the authors used detection and segmentation algorithms to classify four different types of cracks and then segment crack pixels. They [ 46] observe a better pixel segmentation F1-score on the CrackForest dataset than others using a multiple image-resolution training strategy.

Most researchers are focusing on multiple crack classification and having a better a F1-score using an image from a camera with an orthogonal view of the pavement and high ground sampling distances (i.e., pixel per inch). The evaluation of patch-based classification approaches for distress and its severity classification is limited in the literature. Patch-based classification and identification of distress instances are helpful for localized distresses; the technique is suitable for images that capture the top view of the road. It is computationally less expensive than pixel-level segmentation approaches. Table 4 provides a summary of the literature focusing on distress classification using either patch classification, image classification, or semantic segmentation.

Pixel Segmentation Approaches to Distress Detection

Segmentation-based approaches classify or label each pixel as a group or distress. Usually, distinct types of crack distresses are good candidates for pixel-level segmentation. The precise location of a crack can be determined using pixel-level labeling. In [ 111], the authors used U-Net architecture to segment crack pixels using a publicly available crack image database. The number of input training and test images is minimal; the experiment shows promise to segment crack pixels. In [ 112], the authors summarize a review of 68 manuscripts covering deep learning techniques for crack detection using segmentation. The authors evaluated eight segmentation models on 3D pavement images obtained from systems like [ 33]. They observed that FCN [ 113] and U-Net [ 114] performed better than others for 3D pavement images. In another attempt by [ 115], the author proposed a CNN-based segmentation algorithm named DeepCrack. The images are publicly available datasets of cracks from an intensity camera with a top view of the pavements with a dimension of 512 × 512 pixels. DeepCrack architecture, built with different scales and inspired by the SegNet network [ 115]. The authors in [ 100] used VGG-16 DCNN to detect cracks, by dividing high resolution images into smaller patches and use an image classification approach to detect cracks.

The authors have extensively evaluated DeepCrack with other state-of-the-art pixel segmentation models. The experimental result was an average F1-score of 0.85 for DeepCrack. The researchers in [ 116] have investigated the U-Net model architecture for crack segmentation; they used transfer learning techniques on pre-trained weights to train the classifier. The data on the concrete pavement is collected through a mobile phone at various locations at the Huazhong University of Science and Technology, China, with an image dimension of 512 × 512 pixels. The authors claim a higher accuracy and precision for crack pixel classification for concrete pavement types. In [ 117], researchers have proposed an asphalt pavement crack segmentation using a new CNN architecture. The data were collected from 12 cities in Liaoning province, China, through a hand-held mobile phone camera. The researchers have compared the results with existing segmentation models such as U-Net [ 118], SegNet [ 119], PSPNet [ 120], and DeepLabV3 [ 121]. The proposed model performance is better than the existing segmentation CNN architecture. Tang et al. [ 122] proposed an encoder-decoder network EDNet for crack segmentation. The network caters to quantity imbalance between crack and non-crack pixels. The images are taken from the top-view laser scanning camera to acquire 3D pavement images. The proposed method achieves an average F1 score of 97.80% and 97.82%.

Researchers focus more on cracks than other visual distress on the pavement surface when using deep learning techniques. One reason for this is that the crack is a fatigue on the surface that further disintegrates into potholes or total failure of the pavement surface. In Section 2.2, we observe that the occurrence and severity of visual distress, especially cracks, are essential to estimate the pavement conditions index. Table 5 summarizes crack segmentation and detection using deep learning. Researchers have mainly used encoder and decoder convolutional neural network architectures to segment crack pixels. Researchers in [ 94, 107, 109, 123] proposed smaller customized CNN encoder-decoder networks, and the model is trained on smaller patches extracted from the higher-resolution image, while [ 115, 124, 125, 126] used a modified UNET [ 113] based architecture, which is a fully convolutional network for semantic segmentation, and used a smaller resized image. Researchers combined three [ 127] and five publicly available datasets [ 51] for training their models and reported a lower F-1 score than previous ones using three deep learning architectures namely Holistically-Nested Edge Detection (HED), Richer Convolutional Features (RCF) and the Feature Pyramid and Hierarchical Boosting network (FPHB). For crack segmentation, the top orthogonal view is preferred over than front wide view of the pavement. The orthogonal view has the advantage of controlled lighting and higher pixels per inch; however, the disadvantage is of covering a lesser view of the pavement.

We observe that the biases and weights of the encoder (feature extraction part) are trained primarily from scratch instead of pre-trained on the ImageNet benchmark for developing a crack pixel classifier. Most crack detection and segmentation models are evaluated on publicly available datasets. Table 5 shows the test performance (F1- score) of different models developed using different architectures and training datasets. The deep-learning-based algorithms perform well when the test data is similar to the training images (i.e., from the same device); however, the performance degrades when the multiple training datasets are combined, or the test dataset is from a different capturing device and region. We also observe that automated segmentation deep learning algorithms using orthogonal images show a higher F1-score than the front or back view images. Methods like DeepCrack [ 115] holds promise to identify linear (transverse, longitudinal) cracks that are difficult to detect in patch-based methods.

Object Detection Approach to Distress Detection

Distress detection can also be approached using object detection. The approach is somewhat like patch-based image classification; however, the implementation is different in terms of the input and output of the CNN architecture. The object detection method can be used to find multiple object (distress) instances in a high-resolution image using CNN networks like Faster RCNN [ 135], the SSD MobileNet [ 136], or the YoloV3 [ 137]. Object detection-based techniques are usually used to detect different distresses, mainly including potholes, patches, cracks, and their various types and severities (see Table 6 ).

In [ 138], the author proposed a pothole detection system trained on images taken from a hand-held camera. The model was tested and compared with four object detectors. The authors observed that single-shot multi-box detectors (SSD) have higher accuracy but lower computational speed than YoloV3. YoloV3 fails in cases where the size of the pothole is small. In [ 139], researchers used Squeeznet architecture to train a model on image patches of size 64 × 64 extracted from two datasets with an orthogonal view of the pavement. The F1-score using the GAPs dataset was poorer than the F1-score obtained on the custom dataset obtained in the USA. Researchers in [ 45, 140, 141, 142, 143, 144, 145] used a version of the YOLO [ 137] architecture to train a model to detect different distresses. A crack severity detector for the top view of the pavement using YOLO with an average F1-score of 0.70 was proposed in [ 45].

Similarly, in [ 149], the authors experimented with thermal imagery and used object detection algorithms with an average precision of 91.15%. Maeda et al. [ 150] proposed an object detector based on SSD MobileNet and Inception V2 architectures. They achieved an average recall of 77% with a precision of 71% for potholes, alligator cracking, and blurry line marks. However, the 'presence/absence' detection is not very helpful for quantifying distress, which is essential for pavement surface evaluation.

Table 6 shows a summary of object-detection-based distress detection. The F1-scores indicate that the performance of the object detector deteriorates for multiple distress detection compared to detectors that detect one or two distresses. We observe that Yolo architectures promise to detect distresses from a frontal view of the camera; however, developing a robust model for a region will require calibration from the local distresses. Top-view, hand-held cameras, and wide-view images have been used in experiments. The object detection-based algorithm can be used for localized distress detection, such as alligator cracks and potholes. The localization and detection accuracy is better than the patch-based method. Recall or accuracy for detecting cracks (linear or edge) using a frontal view image is less when object detection networks such as Yolo [ 137] are used compared to top-view images.

4.2.5. Automated Direct Pavement Condition Rating

It is highlighted in the introduction of this paper that the primary purpose of distress detection and identification is to evaluate the condition of the pavement using a standardized scale. Distresses must first be identified to compute a rating for an extensive pavement network. Then the number of distinct distresses and their severity must be considered over a given stretch of pavement. Most research focuses on distress identification but falls short of computing a direct pavement rating for a stretch of pavement. One approach to computing direct ratings is described in [ 151], where the authors present a hybrid model of an object detector and semantic segmentation for classifying and quantifying distress severity on pavements and predicted PASER indices for each patch. The images are collected from Google Street View maps - 70-degree wide-angle views, and 90-degree birds-eye view images. Wide-view photos are used for crack and pothole detection, and top-view images are used to quantify crack severity. The results from the hybrid model are then fed to a linear and weighted regressor for predicting PASER indices to pavement patches. They trained YOLO to detect distress and used U-Net (based on a fully convolutional layer) to classify crack severity. The results from the two models are then combined to find the crack density per pavement defect. The results are then fed to a linear and a weightage regressor to label each image a PASER index. The photos are from USA pavements, and the PASER calibration set is minimal. The predicted PASER model fits with an R ² of 0.9382 or test data with a root mean square error of 10.45. One of the limitations of this research is the use of Google API images that are usually older. In this system, only two distresses are taken for the rating (cracks and potholes); however, in most practical scenarios, cracks, potholes, patches, raveling, and bleeding also need to be considered, requiring transfer learning for adding localized distresses further modification in the algorithm for raveling and bleeding.

In [ 152], the authors have presented an image classification approach to surface rating using a three-rating index-good, regular, and bad. The dataset used for the experiments is RTK [ 32], caRINE [ 153], and KITTI [ 154]. It classifies roads into three different types and three different ratings. The images are cropped to extract the region of interest that contains the road. Data augmentation is performed to increase the robustness and avoid overfitting. The authors used three convolutional layers, a flattening layer, and two fully connected dense layers to classify the road types into asphalts, paved, and unpaved. The classified images are then further passed through another classifier to estimate the quality of each road, as good, regular, and bad for each class. The surface type accuracy is reported as 98% for three types. The classification accuracy for the three quality types is 98% for good asphalt and 96% for bad asphalt. The precision of classifying the good class is 86.7%, while classifying the bad asphalt class is 81%. The number of rating indices is limited to three-good, bad, and regular, and they only relate to Brazil's actual standard rating system. However, judging on a scale of 3 levels is not very useful in real life, where maintenance decisions are based on the overall rating, and individual distresses that lead to that rating. Moreover, it also requires further experiments to increase the number of image classes to be adopted for visual standards such as PASER or PSCI. The higher statistics of recall and precision are much easier to obtain if the images are simple-complex images with multiple distress and different quantities are much more difficult.

In [ 155] the author presented the complexity of manual PCSI practices. The author used pixel segmentation using a semantic segmentation CNN-based model from [ 156] to extract roads, marks, and background pixels. They analyzed state-of-the-art EfficientNet V2 [ 157] image classification approach for automating PSCI ratings. Each image in the training and test set has a 'segmented' pavement image, an 'augmented' image, and an 'original' image. Image height is cropped 250 pixels from the top and 50 pixels from the bottom to remove the sky and pavement pixels further away from the camera and pavement pixels too close to the camera. 'Augmented' image is computed by combining the pavement segmented intensity image, the pavement plus mark pixel intensity image, and the original intensity image. They used a combination of these images to evaluate the performance of the classifier. For a 10-class classification, the best model achieved an F1-score of 0.57, while a 0.73 for a five-class classification after combining adjacent classes.

4.3. Benchmarking and State-of-the-Art Models

During the last decade, researchers have developed benchmark datasets to evaluate deep learning models, especially CNN feature extraction layers [ 158], and images labeled for a particular computer vision task. The algorithm is known as a state-of-the-art model if the model's performance matrix is the best if evaluated against benchmarks [ 159]. The website [ 160] gives a structured approach to finding state-of-the-art models for different computer vision, natural language processing, and signal processing tasks on the respective datasets. Improving the state-of-the-art models using benchmark datasets is one approach; however, recently, researchers have argued that an application-centric process must be followed for a deep learning solution. In [ 160], Hooker argues that chasing benchmarks is incorrect for evolving a machine learning model. Instead, smartly chosen training images specific to a particular application helps in better understanding for developing a deep learning-based solution as suggested by [ 161]. Across different subfields of AI, specifically in machine learning, current benchmarking practices tend to distort the development of fair and flexible AI systems for real-world scenarios. In [ 162], the authors systematically explored the limitations of influential dataset-based benchmarks, revealed the construct validity issue, pointed out the risk associated with their framing, and proposed alternative performance evaluation methods. The authors [ 162] have logically argued that the state-of-the-art performance of AI models on these benchmarks does not validate the general-purpose capabilities of models, particularly in visual and language understanding domains.

Therefore, benchmarking is a conservative approach to assessing general model capabilities due to limited task design, de-contextualized data, hidden biases, false performance reporting, and inappropriate community use in the machine learning context. These benchmarks are arbitrarily selected subsets of objects from the real world and cannot cover the domain knowledge for a particular application. It is recommended that along with recalibration or transfer learning with a localized dataset, alternative methods such as unit testing and failure mode analysis could measure the broader capabilities of an automated pavement rating system.

4.4. Limitations of AI-Based Automated Pavement Rating Systems

A country or region's adaptation to a standard, or defining local variants for pavement condition assessment depends on environmental factors, local pavement distresses, and economic factors in the data collection process. The evolution in imaging technology, computational power, and CNNs have made pavement condition assessment through visual distress detection fast, quick, easy, and cost-effective for a country's comprehensive pavement survey. In [ 9], the authors summarized different imaging technologies, types and sub-types of distress, and distinct levels of distress severity (i.e., low, medium, and high). The variation in shape, size, and texture is due to different severity levels of these distresses due to various weather conditions in different geographical locations [ 17]. The variation in data in different regions is not only because of changes in shape, size, and texture of distresses but also due to different imaging technology and placement of sensors in specialized vehicles. Automated condition rating for a pavement stretch depends on types and sub-types (severity-level) of distress and the amount of distress present in a particular stretch. The CNN-based automated decision tools depend on learning from statistical information present in images; therefore, data injected for learning needs to be centric to the problem domain, smartly sized, and less noisy [ 161]. The accuracies and precisions mentioned in the literature are reported on limited data sets, certainly not with complex images with multiple distresses of different shapes, sizes, or textures. Any automated rating system using imaging technology needs to be recalibrated (for example, using transfer learning techniques) for the regional distress to capture variation in shape size, the texture of distresses, and variation in light intensity. The highlighted environmental factors (such as rain, standing water, poor lighting, and moisture) play a crucial role in distress shape, size, and texture. Moreover, while imaging these distresses for pavement condition assessment, the algorithm is not generalizable for different geographical locations due to the distress's environmental factors, shape, size, and texture.

Orthogonal views capturing the pavement requires expensive external 2D and 3D sensors mounted outside the back of the vehicle, which makes it expensive to maintain. It increases the budget for pavement condition assessment for a road network across the country, especially the local network. However, it captures images with controlled lighting conditions, which help in the automated detection and segmentation of cracks and patches. It is recommended for use on national highways and motorways. The frontal view capturing of the pavement requires low-cost cameras that can be mounted inside the vehicle, which makes it less expensive and lower budget. It captures a wide view of the pavement, which helps in the automated detection and classification of different distress types, including raveling, bleeding, different types of patches, cracks, and potholes. It is recommended to cover a bigger network of pavement surfaces, including local and regional roads. Another challenge for such approaches is the unavailability of very large datasets of labeled data-labeled images identifying multiple distress types and their severity levels are expensive to create, requiring both time and expert knowledge.

5. Conclusions

Technology and intelligent algorithms for automated pavement surface condition evaluation have evolved during the last decade. The literature indicates the experimentation in evaluating different imaging technologies (such as intensity, color, and 3D laser camera), imaging road views (top-view, wide-view, or hand-held), and developing a robust algorithm for detecting distinct instances of distresses in an image-moreover, very little work is found on pavement condition assessment rating. The current limitations include a lack of a general evaluation matrix to evaluate the robustness of the detecting algorithms for different shapes, sizes, and textures of distinct distresses in different geographical locations. The lack of algorithms for quantifying these distresses in images and, finally, for rating a stretch of pavement using a sequence of images to develop a real-world automated pavement condition assessment rating. In practice, a rating is assigned to a stretch (200 m or 100 m) of pavement instead of one image; different regions follow different assessment standards.

We found little work on automatically computing direct pavement ratings. The recent literature reviews pavement condition evaluation summarize imaging technologies and different machine learning approaches for distress detection and identification; however, they have limited insight into the correlation between standard condition rating practice to distress detection and its quantification. Road or pavement rating conditions depend on the type of distress and quantification, which changes (shape, size, and texture) with several factors, including environmental conditions (weather) and the pavement construction process. The highlighted environmental factors (such as rain, standing water, poor lighting, and moisture) play a crucial role in these distresses' shape, size, and texture.

For data collection, the top view of the pavement and a wide-angle view of the pavement has been used for distress detection, identification, segmentation, and pavement condition ratings. Choosing an imaging technology for visual inspection depends not only on the type of distress but also on environmental conditions and economic factors and how adequate they are in identifying those distress. The top view gives a higher ground sampling distance but covers less area per image than wide-view images. Vehicles with external laser scanners and stereo pairs are more expensive to operate and maintain than vehicles with an internal high-resolution camera with a frontal view. In summary, there is no off-the-shelf solution for automated pavement condition rating. Most of the existing commercial solutions usually provide automatic data and image-capturing solutions, while their ability to detect and quantify distress from images is limited to a few distresses.

Many of the datasets available as benchmarks are limited only to cracks and potholes and are localized to a geographical location. Research on automated pavement distress assessment is often limited to the regional pavement condition assessment standard that a country or state follows. The criteria a region adopts to make the evaluation qualitative depends on factors such as pavement construction type, type of road network in the area, flow and traffic, environmental condition, and region's economic situation.

Most of the automated image-analysis-based pavement condition assessment focuses on two primary distress, i.e., distinct types of cracks and potholes. Very few experiments can be seen in the literature on raveling or bleeding (see ([ 163, 164]), which are forms of surface defects and contribute toward a unified pavement surface rating. Other surface distress, such as patching, utility patches, and utility cover, is seldom considered (see [ 165]). PASER (used in the USA and other regions) and PSCI (used in Ireland), the ratings 10-7, are decided based on the amount of raveling and bleeding alone. Similarly, the study of direct pavement ratings from images as a classification problem is limited, apart from [ 152] and [ 151]. The 'presence/absence' detection is not very helpful for quantifying distress, essential for pavement surface evaluation. Higher levels of recall and precision are much easier to obtain if the images are simple; complex images with multiple distresses, and their quantities are much more difficult. Automated distress detection and condition rating is not a time-critical process, it can be conducted offline, so accuracy and precision are more important than computational time.

In the future, automatically computing a rating for a stretch of pavement will need to combine several methods. For example, image processing techniques such as cropping may be required to remove objects such as the sky, buildings, cars, and sidewalks to prepare images for use by machine or deep learning models. Then, segmentation may be used to segment the distinct distress and use the number of pixels of each different instance to calculate the area of the distress. A similar approach could be implemented using object detection-based approaches to detect individual distresses. Distresses such as rutting and sag may require multiple images or a fusion of point sensor information to establish the presence of such stresses. Deep learning models will need to be calibrated (trained) to capture the severity levels of each distress for a local region where it needs to be deployed. The number of distresses and their severity can be used to compute a rating score averaged over a set of images for a given stretch of road. Advances in deep learning may allow computing a rating directly using image classification. Still, a lack of benchmark datasets containing various distresses for learning may hinder such approaches. Developing a benchmark dataset for a diverse set of distinct distresses and their severity levels is challenging, as it requires extensive data collection to capture different environmental conditions and regional variations. We propose that any automated rating system for pavement conditions using imaging technologies will require re-calibration (i.e., transfer learning) for the regional distress to capture variations in shape, size, the texture of distresses, and variation in light intensity.

Funding Statement

This project has received funding from E.I. (Grant No. M.F. 2021 0273) and the European Union's Horizon 2020 Research and Innovation Program under the Marie Sokolowski-Curie Co-funding of regional, national, and international programs Grant agreement No: 847402.

Author Contributions

W.S.Q., D.O., D.P., S.I.H., S.M., B.M. and K.F. have contributed substantially to the conception and design, acquisition of data, or analysis and interpretation of data. W.S.Q., wrote the original draft. W.S.Q., D.O. and S.M. have been involved in drafting the manuscript, proofreading, and revising it critically for important intellectual content. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not Applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Publisher's Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

2. PASER Asphalt Roads Pavement Surface Evaluation and Rating PASER Manual Asphalt Roads. 2002. [(accessed on 3 February 2022)]. Available online: http://tic.engr.wisc.edu

13. Arya D., Maeda H., Ghosh S.K., Toshniwal D., Mraz A., Kashiyama T., Sekimoto Y. Transfer Learning-based Road Damage Detection for Multiple Countries. 2020. [(accessed on 25 October 2021)]. Available online: http://arxiv.org/abs/2008.13101

36. Nappo N., Mavrouli O., Nex F., van Westen C., Gambillara R., Michetti A.M. Use of UAV-based photogrammetry products for semi-automatic detection and classification of asphalt road damage in landslide-affected areas. Eng. Geol. 2021 doi: 10.1016/j.enggeo.2021.106363. [[ CrossRef] Google Scholar]

42. Outay F., Mengash H.A., Adnan M. Applications of unmanned aerial vehicle (UAV) in road safety, traffic and highway infrastructure management: Recent advances and challenges. Transp. Res. Part A: Policy Pract. 2020;141:116-129. doi: 10.1016/j.tra.2020.09.018. [[ PMC free article] [ PubMed] [ CrossRef] Google Scholar]

82. Fan Z., Wu Y., Lu J., Li W. Automatic Pavement Crack Detection Based on Structured Prediction with the Convolutional Neural Network. February 2018. [(accessed on 2 March 2022)]. Available online: http://arxiv.org/abs/1802.02208

88. Arya D., Maeda H., Ghosh S.K., Toshniwal D., Sekimoto Y. RDD2020: An annotated image dataset for automatic road damage detection using deep learning. Data Br. 2021;36:107133. doi: 10.1016/j.dib.2021.107133. [[ PMC free article] [ PubMed] [ CrossRef] Google Scholar]

102. Bhatia Y., Rai R., Gupta V., Aggarwal N., Akula A. Convolutional neural networks based potholes detection using thermal imaging. J. King Saud Univ. Inf. Sci. 2019;34:578-588. [ Google Scholar]

110. Ranjbar S., Nejad F.M., Zakeri H. An image-based system for asphalt pavement bleeding inspection. Int. J. Pavement Eng. 2021 doi: 10.1080/10298436.2021.1932881. [[ CrossRef] Google Scholar]

118. Ronneberger O., Fischer P., Brox T. Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015. Volume 9351. Springer; Cham, Switzerland: 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation; pp. 234-241. Lecture Notes in Computer Science. [[ CrossRef] Google Scholar]

121. Chen L.-C., Papandreou G., Schroff F., Adam H. Rethinking Atrous Convolution for Semantic Image Segmentation. arXiv. 20171706.05587 [ Google Scholar]

136. Liu W., Anguelov D., Erhan D., Szegedy C., Reed S., Fu C.Y., Berg A.C. Computer Vision-ECCV 2016. Volume 9905. Springer; Cham, Switzerland: 2016. SSD: Single shot multibox detector; pp. 21-37. Lecture Notes in Computer Science. [[ CrossRef] Google Scholar]

157. Tan M., Le Q. V EfficientNetV2: Smaller Models and Faster Training. 2021. [(accessed on 3 October 2022)]. Available online: https://github.com/google/

164. Ranjbar S., Moghadas Nejad F., Zakeri H. Asphalt pavement bleeding evaluation using deep learning and wavelet transform. Amirkabir J. Civ. Eng. 2022;53:2. [ Google Scholar]

Articles from Sensors (Basel, Switzerland) are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

# data[].attributes.attachment_link.title
An Exploration of Recent Intelligent Image Analysis Techniques for Visual Pavement Surface Condition Assessment

# data[].attributes.attachment_link.description
Lu Xiong, Academic Editor, Xin Xia, Academic Editor, Jiaqi Ma, Academic Editor, and Zhaojian Li, Academic Editor Road pavement condition assessment is essential for maintenance, asset management, and budgeting for pavement infrastructure. Countries allocate a substantial annual budget to maintain and improve local, regional, and national highways.

# data[].attributes.body
https://youtu.be/p6W7xkZFqzo
https://youtu.be/mMvJkVBgatY

# data[].attributes.body
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9029655/

# data[].attributes.attachment_link.readable_text
Nicola Giaquinto, Academic Editor, Francesco Adamo, Academic Editor, and Maurizio Spadavecchia, Academic Editor

Abstract

Road condition monitoring (RCM) has been a demanding strategic research area in maintaining a large network of transport infrastructures. With advancements in computer vision and data mining techniques along with high computing resources, several innovative pavement distress evaluation systems have been developed in recent years. The majority of these technologies employ next-generation distributed sensors and vision-based artificial intelligence (AI) methodologies to evaluate, classify and localize pavement distresses using the measured data. This paper presents an exhaustive and systematic literature review of these technologies in RCM that have been published from 2017-2022 by utilizing next-generation sensors, including contact and noncontact measurements. The various methodologies and innovative contributions of the existing literature reviewed in this paper, together with their limitations, promise a futuristic insight for researchers and transport infrastructure owners. The decisive role played by smart sensors and data acquisition platforms, such as smartphones, drones, vehicles integrated with non-intrusive sensors, such as RGB, and thermal cameras, lasers and GPR sensors in the performance of the system are also highlighted. In addition to sensing, a discussion on the prevalent challenges in the development of AI technologies as well as potential areas for further exploration paves the way for an all-inclusive and well-directed futuristic research on RCM.

Keywords: road condition monitoring, pavement distress evaluation, pavement monitoring, smart sensors, AI, deep learning, machine learning

1. Introduction

Roadway infrastructure is susceptible to structural degradation on account of material deterioration primarily caused by heavy traffic, harsh weather conditions, aging, poor construction quality, and lack of appropriate maintenance. The success of a road transport system is inherently dependent on the riding quality and comfort level of the commuters, for which timely detection of faults and ensuing maintenance is of utmost importance. The lack of rapid and automated road monitoring methods is a major contributing factor to pavement damages and is a common issue in many countries, necessitating huge efforts in this direction. Most developed countries witness a significant portion of aged pavements, while some suffer from deteriorated road networks due to extreme climatic events. On the other hand, developing countries suffer from pavement distress due to a radical increase in the number of vehicles and intensive usage. The manual observation and detection methods, though in practice, are cumbersome and time-consuming, involving high costs. During recent decades, researchers and engineers have prioritized safety and reduction in inspection costs in the scope of developing an intelligent and smart roadway infrastructure system to expedite maintenance and corrective action. Therefore, the significance of cost-effective road condition monitoring (RCM) systems to warrant long-standing structural integrity and safety levels has been emphasized globally. Emerging RCM methods have the potential to rationalize periodic inspections and minimize the costs associated with failing pavement structures. This paper aims to provide a systematic and comprehensive review of the latest advancement in AI-assisted RCM technology.

Pavement failure can be attributed to diverse factors, such as vehicle loading, environmental conditions, construction quality and maintenance. Some of the most commonly occurring pavement distresses are listed in Table 1 . Vehicle loading and environmental conditions majorly contribute to surface disintegration, resulting in potholes, which become severe as disintegration moves down the layers with time. Cracking, which is another common form of pavement failure, occurs in different shapes and sizes, which can again be attributed to vehicle load, causing damage in areas with an inadequate packing of base layers and/or poor drainage, or poorly constructed paving lane joints or volume changes in the subgrade. There are other forms of distresses that come under pavement distortions, such as shoving, rutting, corrugation, depressions, and bumps and sags, which are observed parallel to the direction of traffic and occur due to a weakened pavement layer, poor bonding between layers, or excessive moisture in the subgrade. Surface deterioration is also observed in the form of raveling, weathering and bleeding owing to aging, vehicle loading, poor mix, poor compaction, dust, and moisture. Often, sealants and patches, which are preventative maintenance actions, appear as distresses over the course of time.

Table 1

Distress Type

Severity Levels

Causes

Crack

Pothole and Patch

Pothole

Minimum diameter of 150 mm
Low: depth ≤ 25 mm
Moderate: 25 mm < depth ≤ 50 mm
High: depth > 50 mm

Water infiltration into cracks

Patch

Surface ≥ to 0.1 m ²
Low: rutting ≤ 6 mm
Moderate: 6 mm < rutting ≤ 19 mm
High: rutting > 19 mm

Lack of preventative maintenance

Surface Deformations

Surface Defects

A typical RCM method relies on the data acquisition system that involves 1D time-series data, 2D visual data or 3D depth data. Low-cost sensors, such as accelerometers, gyroscopes, magnetometers, and GPS, are often employed to measure motion, rotation, velocity, orientation, and location, which are one-dimensional in nature. The commonly used sensors for acquiring digital images are the high-quality, high-resolution RGB sensors, which facilitate in acquiring multiscale low-level as well as high-level feature extractions. However, these images are two-dimensional in nature and cannot be used for the characteristic evaluation of the features. The depth consideration of such images allows multi-fold parametric attributes facilitating the examination of intrinsic characteristics and can be acquired by using thermal imaging sensors, LiDAR, laser sensors and GPR, to name a few. Table 2 provides a comparative evaluation of the next-generation sensors that have evolved the entire data acquisition process for RCM. The convenient deployment of these sensors on the ground and aerial platforms (i.e., drones) provides them with an edge in terms of scalability and employability. Table 3 illustrates the merits and limitations of different platforms on which sensors are commonly deployed.

Table 2

Key Variables

Camera

Laser

GPR

Thermal

Vibration

Table 3

Sensor Platform

Advantages

Limitations

Unmanned Aerial Vehicle

Large FOV.
High Resolution.
In-depth, detailed data.
Ease of deployment and accessibility in hazardous areas.
Flexibility for quick inspections.

Payload and memory restrictions.
Legislative restrictions.

Ground Vehicle

Long span availability.
Array of sensors.
High-resolution imagery.
Highly dense and occluded terrain.

Small FOV.Less cost-effective.
High dependency on manpower.

Smartphone

Lightweight technology.
Economically viable.

Low-resolution imagery.
Limited by RGB data.

Advances in next-generation sensors [ 1] and data-driven technologies [ 2, 3, 4] have transformed data collection and interpretation, contributing to extensive research on RCM. Recent years have seen the widespread use of vision-based approaches to provide cost-effective solutions in identifying and localizing pavement anomalies. Diverse methods have been presented, ranging from traditional image processing techniques to vision-based AI approaches. The archetypal vision-based RCM is commonly carried out either by employing image processing techniques (IPTs), such as thresholding, or by using machine learning (ML) algorithms for local feature extraction. In the process of implementing IPTs, encouraged by grayscale and texture-based methods, Ref. [ 5] performed asphalt pavement pothole detection and segmentation using a wavelet energy field to highlight the pothole region. The proposed method exhibited satisfactory performance with an overall accuracy of nearly 87%, and successfully differentiated the potholes from the cracks, patches, greasy dirt, shadows, and maintenance hole covers, and precisely segmented the pothole. However, the overall methodology was time consuming and required tedious processing.

Ref. [ 6], in his attempt to classify a distressed area, developed a unique two-stage RCM system. In the first stage, the artificial bee colony (ABC) algorithm was used to obtain a threshold value to segment the image into distressed and non-distressed sections, followed by feature extraction. The second stage involved training on a single-layer artificial neural network (ANN). Although the proposed method successfully classified an area of distress, there was a redundant and computationally intensive step to segment the images using the ABC algorithm, as the neural network alone can be efficiently trained to segment the images and extract features for classification. Ref. [ 7] proposed an unsupervised and automated pavement crack detection based on the analysis of photometric information from two-layer (intensity, depth) images. The method is based on histograms and Otsu's thresholding. The results showed that the proposed method could be used for fast and approximate crack estimations, especially in cases of a low signal-to-noise ratio. Despite the limitations of the conventional IPTs in terms of handling the background complexities and illumination complications, continuous research has been carried out using IPTs to enhance and amplify the results. Recently, Ref. [ 8] proposed a three-phased crack detection system based on IPTs. The approach involved contrast enhancement followed by the application of a discrete wavelet algorithm for effective detection and, finally, Jerman enhancement to enhance crack detection. Comparisons with existing methods showed the effectiveness of the proposed technique to validate the recognition of surface cracks.

To improve the accuracy of existing RCM methods, ML approaches, such as KNN, SVM, and k-means clustering, were integrated with IPTs. Ref. [ 9] used ML-based pattern recognition methods with computer vision methods on street view images for the assessment of pavements. A series of IPTs followed by SVM [ 10] classification successfully identified the severity and location of pavement defects in the image. A comparison with the U-Net DL model on the same dataset suggested inferior performance due to scarce data. Ref. [ 11] presented a model based on the library of SVM (LIBSVM) to distinguish the potholes from the cracks in concrete pavements. The proposed model is effective in segmenting potholes with a high F1 score of 98.7%. A comparison with the Otsu, edge detection, k-means, and watershed methods re-establishes the processing efficiency of the method on cement concrete pavement potholes.

Ref. [ 12] adopted a k-means clustering algorithm followed by Otsu's thresholding to segment cracks in asphalt pavements. The proposed algorithm provided a satisfactory result in localizing and detecting pavement cracks. Ref. [ 13] developed a texture-independent, tile-based image-processing algorithm to detect pavement cracks and classify them into longitudinal and transverse cracks on 2D and 3D pavement images. The lengths of the cracks were measured using curve fitting and an orientation axis. The method suggested promising results when tested on 130 images of Portland cement concrete and asphalt concrete surfaces, and took less than 20s per image to generate results. Ref. [ 14] adopted an integrated approach, establishing IPTs in association with ML algorithms for automated road crack detection and classification. Heuristic segmentation and denoising algorithms were applied, followed by image-processing practices for enhancement and feature extraction, and the application of a multi-stage hybrid model based on ensemble learning for classification. A comparative study with existing ML approaches underscored the competitive performance of the proposed work.

In the past few years, there have been several review papers published in the field of RCM. For example, Ref. [ 15] performed an extensive review on the RCM methods focusing largely on the types of pavement distresses and the detection equipment used. A quantitative analysis of the work conducted in the past on individual pavement distresses, such as cracks, potholes, surface deformations and defects, and the macrostate of the road, such as road roughness, skid resistance, substructure quality and road surroundings, were performed. Different equipment, such as a camera, laser, accelerometer, acoustic, pressure sensor, radar, ultrasonic, deflectometer, friction tester for distress detection and data collection, are emphasized and compared in the study. The paper presents a detailed discussion on the various distresses and detection techniques; however, it does not focus on the algorithms used for RCM. Another substantive review by [ 16] highlighted the advancement of deep learning (DL) methodologies in accurate pavement crack detection. The DL techniques were catalogued under three headings: classification, object detection and segmentation. The performance metrics employed to evaluate these methods on benchmark datasets in addition to traditional and DL-based crack-detection methods on 3D data were also reviewed. In another review paper, Ref. [ 17] assessed the various computer-vision techniques employed in the detection of cracks and potholes in pavements. The techniques were categorized under DL, non-DL, depth camera and tire pressure/vibration methods. Varying inputs, such as RGB images, laser and thermal images for vision-based techniques, were discussed and evaluated. It was deduced that major detection techniques were oriented around computer vision approaches, though they failed to produce good results. RGB images owe their limitations to lighting conditions, texture variation and background complexity; however, laser and thermal imaging overcome these limitations, but suffer due to environmental and cost factors as well. Limitations in terms of the system size, computational requirements and real-time applicability were also highlighted in the study. Unlike DL methods, [ 18] in their review, summarized the research carried out on digital images in RCM based on IPTs and acquisition devices. The image acquisition devices, along with their characteristics, were compared, and various image technologies applied in pavement detection were expounded. Concurrently, the IPT for specific problems, such as on-site pavement cracks, pavement texture, rutting and potholes, was elaborated. The authors concluded that the salient factors in determining the detection equipment should be based on cost, efficiency, clarity of image details and type of pavement, while ML methods outperform the traditional IPT in RCM.

Over the last few years, vision-based approaches have augmented RCM solutions and remarkably advanced the development of pavement monitoring and analysis. It is imperative to note that there are other existing reviews [ 15, 16, 17, 18] addressing pavement distress studies, as discussed above. These review articles largely emphasize the types of pavement distresses, the detection equipment used, the sensor technologies available, and various computer vision techniques employed in the detection of cracks and potholes in pavements; however, they do not focus on the data-driven algorithms used for distress detection. The review on pavement-defect detection methods based on DL by [ 16] provided a substantive overview of the subject, but has a restricted scope. The proposed review in this paper aims to provide an all-inclusive review by focusing on the next-generation sensing technologies and associated AI-based RCM methods, by elucidating the methodologies and challenges in current developments, as well as recognizing the prevailing research voids for further research studies. Therefore, the criteria for consideration of research articles in this paper are the evaluation of the existing sensor-based and AI approaches deployed on different platforms, namely UAVs, ground vehicles and smartphones for RCM. AI is a data-driven amalgamation of various ML and DL algorithms inherently dependent on sensors and data acquisition to provide solutions to real-world problems. These algorithms are a subset of the entire AI domain and have been shifted from handcrafted feature extraction-based ML methods to automated DL methods. To help researchers and engineers better understand the application of AI methodologies in pavement monitoring and analysis, the current review summarizes the recent work that has been established from the year 2017 to 2022. The AI approaches are classified under sensor-based methodologies, along with the application of machine learning and deep learning algorithms for RCM. An exhaustive review of the DL methodologies is presented that builds on classification, segmentation and detection, employing next-generation sensors integrated with different data acquisition platforms.

This paper is organized as follows. Section 2 presents a brief discussion of the reviewed articles and their limitations as a comprehensive and systematic review. It elaborates on AI technology in general, categorized under next-generation sensors deployed on data acquisition systems and compares the related work in terms of the methodology adopted and results achieved. Section 3 presents the potential future research directions. Conclusions and challenges in using DL approaches in RCM are finally presented in Section 4.

2. Smart Sensing of Pavement Distress Data

Smart sensing techniques provide high-resolution information on pavements. Faster installation and flexibility in deploying noncontact sensors on various mobile data acquisition platforms facilitate convenient ways to inspect pavements, overcoming space and environmental constraints. Every platform has its advantages and limitations and complements each other's usability. A schematic representation of the sensors and data acquisition platforms is shown in Figure 1 .

Smart sensors primarily result in 1D (i.e., vibration time series) and 2D/3D (i.e., vision) RCM data, which were analyzed using various signal processing, machine learning, and AI techniques in the literature, as presented below.

2.1. Vibration Data-Based RCM

Smartphones are installed with diverse sensors, such as gyroscopes, accelerometers, and GPS receivers, and thus act as powerful measuring tools. Hence, capitalizing on the in-built vibration sensors to assist RCM has taken center stage in numerous research works. Vibration-based methods for detecting road anomalies rely on 1D data acquired from these sensors and find high applicability in the real-time detection of pavement distress due to low cost, fewer memory requirements and immunity to light conditions.

Ref. [ 19] used smartphone accelerometers to measure acceleration data. Basic signal processing and pattern recognition methods were used to correlate the measured data with the international roughness index (IRI), validating that variance in acceleration measurements can be used to determine IRI using smartphone sensors. Ref. [ 20] proposed an asphalt pavement quality monitoring system using data collected from accelerometer sensors of smartphones by treating the data as a multi-dimensional time series classification problem. The proposed approach attained a good classification accuracy. Ref. [ 21] compared a standard inertial profiler with a smartphone-based application, Roadroid, to establish the usability of smartphone sensors in estimating the ride quality of the concrete pavement. It was observed that Roadroid significantly underestimated the IRI values, though a linear correlation existed between the IRI measurements obtained from both methods. In another attempt, Ref. [ 22] proposed an AI-assisted smartphone-based data-driven technique to detect vibration-induced road anomalies using vibration sensors and the detection of pavement patch defects using smartphone images. The results showed that vibration-based methods were quite successful. However, they failed to efficiently cover the entire roadway and detect non-vibration-induced pavement anomalies, which were then complemented by a vision-based method.

Ref. [ 23] presented an AI-based low-cost RCM system by detecting pavement potholes using smartphone sensors and onboard diagnostic devices. The method is based on the observed patterns of the vehicle's interaction with the road surface and confirms the usage of smartphone sensors in low-cost pothole detection. Ref. [ 24] provided a cost-effective pavement evaluation system for low-volume roads using smartphone-based roughness data. Compared to conventional roughness measurement methods, it was found that a good correlation existed between the results, when compared to the results from the Class III-type roughness measurement equipment, justifying sufficient accuracy to determine the presence of pavement distresses. Ref. [ 25] developed an RCM technique by using 3D pavement data to train ML models on low-cost vehicle-mounted smartphone sensor data. The model provided distress values as outputs, which can be used for estimating IRI. The applicability of the technique could be justified by a high peak in the correlation between IRI estimations produced from numerous runs along the same route. In another study, Ref. [ 26] proposed a method to determine the IRI of a pavement surface using conventional vehicles and smartphones by using the grey box model algorithm and the quarter-car vehicle model. The results indicate that the IRI of the pavement can be determined with reasonable accuracy using smartphones.

In a novel attempt, Ref. [ 27] proposed a vibration-based RCM system by considering factors, such as vehicle suspension and a direct reconstruction of the pavement profile, to compute IRI rather than using correlation-based procedures. Field testing of the method proves its potential for RCM with reasonable accuracy and efficiency. Ref. [ 28] used low-cost smartphone accelerometer data by developing a prototype smartphone application to measure the IRI, by incorporating vehicle speed and vehicle type. The results showed a strong correlation with the measured IRI data collected using profiler vans. Ref. [ 29] proposed a low-cost method employing smartphone sensors for pavement roughness measurement by collecting the vertical vibration values when a vehicle is driven over a pavement. The smartphone-based acceleration values are converted to the IRI by using the quarter-car simulation model. The high correlation values promise a model that can be used in smartphone-based RCM. Ref. [ 30] proposed a road condition tool (RCT) to assess road pavement defects based on a telemetric data crowdsourced by smartphone users by analyzing the vehicle motion dynamics. The results obtained, in comparison with reference data from highly specialized equipment, confirm the proposed RCT solution [ 31] established an algorithm to calculate the IRI from the acceleration values obtained using smartphone sensors. The algorithm identified the physical parameters of a quarter actual vehicle model and established a relation between the acceleration, IRI and the profile elevation. It was observed that a consideration of the dynamic characteristics of vehicles improved the proposed method's measurement accuracy.

Thus, vibration-based methods find immense applicability in RCM. They provide a comparative measure of the degraded pavement condition that is memory friendly and suitable for the real-time detection of road anomalies, but are vulnerable to errors due to noise and signal sensitivity from other vehicle sensors. Moreover, the vibration effects caused by any road obstacle can be perceived as a road anomaly by these sensors. Additionally, it is important to note that vibration-based sensors only take into account data along the wheel path.

2.2. Vision Data-Based RCM Using Machine Learning

The exploitation of ML methods allows for the effective identification, classification and analysis of pavement structure conditions. Generally, ML approaches in pavement engineering comprise support vector machines (SVMs) and artificial neural networks (ANNs). The following section briefly summarizes the conventional ML approaches used in recent years in RCM using different data acquisition platforms.

2.2.1. UAV-Based Data Acquisition

Multispectral image-based ML algorithms were employed by [ 32, 33] on drone-acquired images to distinguish undamaged pavement from damaged pavement. SVM, ANN, and Random Forest (RF) algorithms were evaluated and compared both on RGB and multispectral images. It was deduced that, amongst the three algorithms, RF performs best with a high accuracy on multispectral images. In addition to the feature set, it was deduced that the spatial resolution of the pavement imagery is also a conclusive factor in the performance of the classifier. Ref. [ 34] developed an RCM system based on drone images using image-based methods to identify three types of cracks in pavements. Decent results were achieved by the proposed combination method developed by combining Canny edge detection and Otsu thresholding. Ref. [ 35] proposed the usage of low-flying UAV acquired multispectral imagery in combination with CNN extracted spatial features to assess the aging and damage conditions of pavements with the help of an SVM classifier. A multiscale semantic segmentation algorithm was carried out on the UAV pavement imagery for classification. Three principal components obtained, after dimensionality reduction in the multispectral imagery, were combined with deep abstract spatial features from the CNN and fed as inputs to the SVM classifier for pavement surface classification into cracks, potholes, early aging, middle aging and late aging. However, the dimensionality reduction in multispectral imagery demands additional processing. Moreover, low-flying UAVs may not be a practical approach for dense-traffic roadways.

2.2.2. Smartphone-Based Data Acquisition

Ref. [ 36] carried out an exhaustive review on road surface anomaly detection using smartphone sensors, while highlighting various issues and challenges in the current approaches, and identified research gap areas in the domain. The factors affecting anomaly detection, such as smartphone orientation, mounting location, vehicle suspension and speed dependency, were also investigated. The existing approaches were compared using five primary criteria: sensor data collection, pre-processing, processing, post-processing, and performance evaluations. The major focus of the review was on data collection and processing, rather than result-oriented methodologies, to perform anomaly detection. Ref. [ 37] performed a smartphone-based pavement analysis to classify the road surface conditions using an accelerometer, gyroscope and GPS data. Multiclass supervised ML techniques were applied using features from all three axes of the smartphone sensors. The classification was focused on three main classes- smooth roads, potholes, and deep transverse cracks. The results confirmed a higher performance than models using single-axis features.

2.2.3. Ground Robot-Based 3D Data Acquisition

In contrast to the traditional 2D and manual methods, Ref. [ 41] proposed a 3D laser-based pavement scanning and automatic defect detection approach to identify micro (cracks) and macro (deformation) defects. A sparse processing algorithm to extract crack candidate points and deformations support points were designed. The algorithm successfully obtained an accurate location and the classification information of defects with a detection accuracy above 98%. Ref. [ 42] defined an ML-based system for the acquisition and intelligent classification of potholes and cracks in pavement surfaces. A Kinect device was used to obtain 3D point-cloud structures of pavements and retrieve their physical properties. The system could successfully classify cracks and potholes, along with the computing of the physical properties of pavement depressions, such as the length, width and depth. Ref. [ 43] created a method for detecting road damage by using a laser scanner mounted on a mobile mapping system to generate 3D pavement images. An attempt was made to segment the pavement using a series of ML techniques. Ref. [ 44], in their work, developed a computationally efficient pothole-detection algorithm based on disparity maps and disparity transformation algorithms. The detection of potholes was finally achieved by a comparison between actual and modelled disparity maps. The point clouds of potholes were extracted thereafter. The authors created three datasets for stereo vision-based pothole detection, and the algorithm achieved an overall accuracy of 98.7%.

Ref. [ 45] proposed a fully automated LiDAR-based, two-fold RCM assessment over highways and airport runways using a mobile mapping system. Firstly, a fully automated algorithm was proposed to detect and locate pavement distress from the 3D point cloud, followed by characterization by severity in terms of the depth below the road surface, surface area at different depths and filling volume of the detected potholes. The accuracy of the point clouds was determined to be ±1-2 cm and elucidates the capability of LiDAR-based mobile mapping systems to precisely detect locations with pavement distress of varying severity and different causes. However, the entire setup of the mobile mapping system turns out to be an expensive affair. Ref. [ 46] developed a pavement evaluation method using mobile mapping systems (MMSs) for automatic detection by spatial frequency analysis of 3D point-cloud data acquired by laser scanners mounted on MMSs. Pavement distresses, such as rutting, flatness and potholes, were detected, along with the crack percentage. The choice of platform and sensors is totally dependent on the specification of a task at hand, taking into consideration the scope, frequency of operation, area in question, extent of detail, budget and strategic importance. A comparative summary of each platform's usability in the field of pavement distress evaluations is listed in Table 4 .

2.3. Vision Data-Based RCM Using Deep Learning

DL methods have become the most extensively used computational approach in the field of civil engineering and intelligent transportation systems [ 47, 48], attaining outstanding results in RCM. One of the benefits of DL is its ability to learn large quantities of data. With the rapid development of DL techniques, deep convolutional neural networks (DCNNs) have gained paramount significance in carrying out vision-based tasks. In comparison to the conventional handcrafted feature-based techniques, the DL-based techniques learn multi-level image features in detail, which are more descriptive than the handcrafted features. A typical CNN architecture is created using several layers, such as the input layer, hidden layers comprising convolution, pooling and activation functions resulting in feature maps, and the output layer. A deep CNN is characterized by an architecture composed of many layers. Auxiliary layers, such as dropout and batch normalization, are also incorporated within the aforesaid layers as per the necessity of purpose. A schematic representation of a CNN is shown in Figure 2 . The first layer of a CNN is the input layer, which is a 2D representation of the input image. The convolution layer performs an element-by-element dot product between a subarray of an input array and a receptive field, followed by a summation of the multiplied values to which the bias is added. This layer is responsible for the reduction in the input data size and subsequent reduction in the computational cost. Batch normalization is performed to re-center and rescale the inputs by normalizing the data distribution. It plays a significant role in smoothening the loss function by optimizing the model. The activation function imparts nonlinearity to the model for semblance to complex real-world problems. The pooling layer is another significant aspect of the CNN responsible for down-sampling as it reduces the spatial size of an input array. A dropout layer can be added to the CNN as an optimization layer to handle complex models that might be prone to overfitting. All these layers, excluding the input layer, comprise the hidden layers of the CNN whose output is fed to fully connected layers resulting in the classification and detection post-application of a suitable activation function.

In general, vision-based DL methods undertake three major pattern recognition tasks: classification, object detection and image segmentation. Classification determines the presence of objects in an image or video. It refers to training DL models to find the classes (pavement distress types) that are present. Classification is useful at the binary level of decision making, whether an image contains the desired object/anomaly or not. A distinct task from the classification is localization, which determines the position of the classified object. Object detection combines classification and localization to determine the objects that are present and specify where they are in the image. It applies classification to distinct objects and uses bounding boxes to show the detection and location of an object. Image segmentation divides an image into regions, extracting potentially meaningful areas for further processing, dependent on its shape and border, such as classification and object detection. The goal of image segmentation is to highlight foreground elements and make it easier to evaluate them. Image segmentation provides pixel-by-pixel details of an object, making it different from classification and object detection. Thus, object classification identifies the category of objects in the image, object detection identifies the category and location of objects with rectangular bounding boxes, and segmentation predicts the categories of each pixel and distinguishes the object instances.

In this paper, the literature review in the following section is catalogued under DL as classification, segmentation and distress detection, and further sub-categorized under data acquisition platforms. It is important to emphasize here that DL-based classification and segmentation methodologies in recent years were restricted to data acquired using ground-based sensors, while UAV and smartphone-based data acquisition platforms find major applicability in areas related to distress detection in RCM, rather than classification and segmentation.

2.3.1. Classifications in RCM Using DL

An autonomous real-time road crack and pothole detection algorithm was proposed by [ 52], deployable on a GPU-based conventional processing board with an associated camera. The approach was based on a deep neural network architecture integrated with a pre-processing method to ensure real-time performance. The authors assured that the proposed system could be installed as a plug-and-play module in various autonomous robots and self-driving cars. Ref. [ 53] proposed the application of CNN on Ground Penetrating Radar images for automatically classifying pavement subgrade defects. Two CNNs called multi-stage CNN and cascade CNN were established to accomplish the tasks that achieved high accuracies. A comparison with ML algorithms further confirmed the CNNs' robust performance. However, despite the 3D images, the physical characteristics associated with the defects were not calculated. To maximize the evaluation accuracy, an automated pavement evaluation system was proposed by [ 54]. Two-channel images obtained by overlaying gray-scale (2D) and range (depth) images were fed to a CNN network for image classification. Though the accuracy metrics were not specified, the authors claimed the superiority of the method over conventional ML techniques. Ref. [ 55] presented an enlarged dataset GAPs v2 as an extension of the GAPs dataset by adding 500 images and providing more refined annotations. Additionally, a CIFAR-like subset of 50 k images was also created for easy comparison and evaluation. Extensive experiments on the extended GAPs dataset were performed using existing DL architectures to understand the role of context in distress detection. Ref. [ 56] proposed an image-based system for the classification of a speed bump and pothole using a five-layer simple convolutional neural network. The proposed network achieved a classification accuracy of 97.7%.

Ref. [ 57] proposed the application of CNNs for pothole detection. A pre-pooling CNN was designed, which added a pre-pooling layer before the first convolution layer to improve the precision of pothole detection. The results demonstrated that the optimized pre-pooling CNN had 98.95% recognition precision on testing data. The stability and comparative study revealed the robustness of the CNN model in real-world situations, such as varying light conditions and pavement materials. Ref. [ 58] suggested an edge AI-based framework named the vehicular ad hoc network (VANET) for road anomaly detection, such as a pothole, bump, and cracks. Pre-trained DL models, VGG-11 and ResNet-18, were used parallelly for the automated classification and detection of anomalies. The dataset employed was collected from various online sources, which was augmented for better results, and it was suggested that the model's performance superseded the other techniques used for the detection and classification of road anomalies. Ref. [ 59] proposed a method for automatic pavement evaluation by the identification, classification and quantification of multiple urban flexible pavement distresses through the application of CNNs. The process involved the concatenation of two CNNs, one that performs identifications with reasonable precision, recall, and F1 score, followed by the quantification of the severity of the predicted distress. Ref. [ 60] applied a three-layer DCNN to implement crack detection in asphalt pavements by resizing images and classifying them as crack and non-crack. The DCNN was evaluated on two benchmarked datasets and a self-collected dataset, with high accuracy on all datasets.

2.3.2. Segmentation in RCM Using DL

Ref. [ 64] presented a Feature Pyramid and Hierarchical Boosting Network for pavement crack segmentation and a novel crack measurement technique. The crack detection was formulated as a pixel-wise binary classification task. The method, along with other ML and DL algorithms, was compared and evaluated in relation to five datasets to demonstrate the superiority and generalizability of the method. The authors also presented a pixel-annotated crack dataset of asphalt pavements named Crack500. Ref. [ 65] presented a road crack segmentation method based on generative adversarial networks (GANs) based on U-Net [ 66]. The GAN [ 67] is a special type of neural network architecture used to discriminate between real and fake images, until it is unable to distinguish the real from the fake. The evaluation of the model was performed on three datasets, of which two were public, and one was customized. The model performed moderately on the public dataset with a maximum F1 score of 77.3% and did not obtain good segmentation on a custom dataset. Ref. [ 68] implemented two DL networks, an improved CNN with structured prediction and an FCN, to detect cracks on pavements with different severity levels. The classification of the cracks was based on crack severity, with an F1 score of less than 70% for both methods. Moreover, the Laser Crack Measurement System software was used, which employed laser intensity to compute the depth of the cracks. The depth factor played a role in determining the severity level during the annotation process.

An attempt to automate pavement distress classification based on pixel-level segmentation was performed by [ 74] using U-Net and Resnet architecture. A dataset was prepared from over a 350 km stretch of asphalt pavements in China along with data augmentation, resulting in over 10,000 images of six types of distresses. Binary classification (distress or non-distress) was performed, resulting in a high accuracy of 97%, while the multi-classification accuracy was fairly low. Ref. [ 75] proposed the Mask RCNN [ 76] approach for the pixel-wise segmentation of the Region of Interest in pavements for pothole detection and area calculations. The database was manually collected and annotated with an overall accuracy of 90% for a computed area of a pothole. Ref. [ 77] proposed a novel DL segmentation approach based on synthetic training data generation for the segmentation of cracks in road pavement images. The synthetic data repository was created using three publicly available datasets: KITTI and Cityscapes for the images of road scenes, and Crack Forest Dataset, as a source of pixel-level labeled cracks. Segmentation was implemented using Mask R-CNN and U-Net, showing good results.

A grid-based crack detection model was proposed by [ 78] using a segmentation CNN approach. Each image was divided into three grids of different sizes, namely 10 × 10, 20 × 20 and 30 × 30, as inputs to the segmentation model to classify crack or no crack. Grids with cracks were segmented into binary images using the entropy thresholds, which were then used to calculate the crack widths. Recently, Ref. [ 79] performed an extensive study by incorporating stereo vision and DL to efficiently perform crack and pothole segmentation using an automated pixel-level detection framework for RCM. The multi-view stereo imaging system was used to establish the datasets containing color images, depth images and color-depth overlapped images. A modified U-Net was implemented and tested on asphalt roads with millimeter-level accuracy. The 3D crack segmentation model outperformed the other models in terms of the inference speed and segmentation accuracy. Additionally, a highly accurate automated pothole volume measurement method based on a segmentation map was proposed to evaluate pothole severity.

2.3.3. Damage Detection in RCM Using DL

Ref. [ 80] proposed the use of the Faster R-CNN algorithm on a customized dataset for damaged pavement detection, which was divided into six distinct categories: lateral crack, longitudinal crack, pothole and separation, alligator crack, well cover without damage and damage around the well cover. The proposed CNN architecture showed high accuracy and stability in locating and identifying the cracks in the dataset. Banking on the advantages of automatic feature extraction by CNNs, Ref. [ 81] presented a framework based on DL to automate the roadway condition assessment. A CNN was designed to classify four types of pavement cracks, namely, longitudinal, transverse, alligator and potholes, and the crack detection dataset was created from images taken from search engines. The overall accuracy of the proposed CNN was 76%, which was justified as reasonable when compared to the methodologies limited to binary classifications.

Ref. [ 82] analyzed the feasibility and accuracy of thermal imaging in the field of pothole detection. The work involved the acquisition of thermal and visual images of potholes under various conditions and the application of augmentation techniques, followed by the implementation of convolutional neural networks on the images. A comparison between the customized and pre-trained models revealed the best accuracy of 97.08% with ResNet101. Ref. [ 83] implemented a novel method for pothole detection by integrating images with mobile mapping point-cloud data. This method first detected the location and 2D edge of a candidate pothole in the image. The point clouds were then used to differentiate between potholes and patches by calculating the mean depth of the candidate pothole. The geometric accuracy of the pothole extraction was evaluated using a simulation experiment, showing that the mean size accuracy was ~1.5-2.7 cm.

Ref. [ 84] utilized existing DL methods to carry out a hotspot analysis on urban road networks, highlighting pavement distress types and associated severities in terms of remedial actions that were either necessary or not. Efforts were also made to outline approaches to using the analysis for the continuous monitoring of pavement health. Damage detection models were accurately able to highlight the location and assess distress severity. However, the severity analysis was limited by human bias carried out during data annotation, rather than post-detection employing mathematical calculations. To assess the workability of DL models on Google Street View images, Ref. [ 85] implemented two classical DL frameworks for pavement condition assessment on a customized labeled dataset. The images were manually annotated, and the distresses were classified into nine categories, namely, reflective, transverse, block, longitudinal, alligator, sealed transverse, sealed longitudinal, and lane longitudinal cracking, along with potholes.

Ref. [ 86] developed stereo-vision techniques for the automatic identification of potholes. Transfer learning-based DL methods for segmentation (Mask R-CNN) and localization (YOLOv2) were also used for pothole detection, suggesting good precision results and real-time applicability, respectively. Ref. [ 87] proposed a 2D vision-based pothole detection method using two-stage CNNs that focused on the discriminative regions in the road instead of the global context. The approach was based on using two CNN subnetworks, a localization network to find the regions likely to contain potholes based on heat maps and a second CNN for classification, which feeds on the candidate regions from the localization network. The experiments on the pothole dataset showed that the proposed method achieved high precision and recall, and outperformed most existing methods. However, this approach turned out to be expensive as the localization network added prior to the classification increased the computational time.

Ref. [ 91] proposed a solution to detect dry and wet potholes using smartphone sensors and camera images. Twelve different sensor values were collected and manually annotated to perform pothole detection using ML algorithms. The captured images were examined for potholes by applying Mask R-CNN and U-Net segmentation algorithms. Both sensor and camera-based methodologies provided promising results. Ref. [ 92] proposed a RCM system that can be integrated with road-damage acquisition systems and presented an asphalt pavement dataset with over 45,000 instances of various distress types. RetinaNet architecture with VGG16 as the backbone was used, promising a better performance than other object detection models. The light memory footprint allows the model to have an easy integration into mobile systems.

Ref. [ 93] presented an RCM method based on Faster R-CNN to recognize and locate cracks, potholes, oil bleeding and dot surface autonomously. A total of 20 Faster R-CNNs were trained and tested on 6498 pavement images, with the performance of the optimal one having accuracy rates, recall rates and location errors of 90.4%, 89.1% and 6.521 pixels. In comparison to the CNN and K-value method, the optimal Faster R-CNN located pavement distresses with more precision [ 94] proposed a methodology for automatic pavement image distress detection and classification using CNN models and a low-cost vehicle-mounted high-definition camera. The pavement distress types were categorized as linear or longitudinal crack, network crack, fatigue crack or pothole, patch, and pavement marking. The detection rate and classification accuracy of the proposed approach with the trained CNN model reached 83.8% over the test set. A sensitivity analysis was also carried out for evaluating the different regularization techniques and data generation strategies.

In view of inadequate large-scale data in civil engineering projects, Ref. [ 90] presented RDD2020 by extending the existing RDD2018 dataset using GAN-based data augmentation. PG-GAN and Poisson blending were used to generate realistic road damage images, which fared well in the visual Turing test. The pothole dataset was increased to 2000 images, and DL methodologies performed reasonably well on the augmented dataset. Ref. [ 98] proposed a DL methodology to automatically localize diseased areas in pavements using a novel approach named the Iteratively Optimized Patch Label Inference Network. The method involved training image patches obtained by slicing high-resolution images and inferring the patch labels using an algorithm based on the label of the high-resolution image. The authors also released a novel, large-scale RCM dataset, CQU-BPDD, involving various pavement diseases.

Ref. [ 99] implemented a highly accurate pavement defect and cleanness inspection system using a DL-based framework in the pavement-sweeping robot Panthera. A lightweight DCNN model was developed and trained on 6000 pavement defect and garbage images, which took approximately 132.2 milliseconds for detecting both pavement defects and garbage. Moreover, the geotagging of the pavement defects helped in mapping the defects. In their work, Ref. [ 100] suggested that pothole detection categorized on specific road types produced better results. The proposed model first classified the road surface as asphalt, unpaved and paved using a CNN algorithm, followed by three YOLO v3 models, respectively, for different road surfaces to detect potholes. Public datasets were used for training, and augmentation was performed to improve the model accuracy. The authors went a step further by geotagging the images during detection and updating a database to mark the potholes on maps' pothole tracking. The geotagged images were used to identify the street and keep an account of distress conditions in that area.

Ref. [ 101] presented a CNN-based pothole detection method on a small dataset and evaluated the performance of Faster RCNN with YOLOv3. As per the results, Faster RCNN performed better than its counterpart. The accuracy of Faster RCNN was further improved by integrating it with feature extraction using VGG16. The dataset was also increased using augmentation techniques to achieve a better performance. Ref. [ 102] compared and evaluated various DL object-detection techniques for damaged pavements to detect potholes. The Kaggle and RDD2020 datasets were used for comparison, and it was concluded that YOLOv4 performed the best with a mAP of 0.535. Ref. [ 103] proposed a CNN-based approach for the automated detection of pavement distresses using images collected from smartphones. The distresses were classified into six categories [ 90]. The system demonstrated a practical efficiency and detection rate with more than 80% accuracy. Ref. [ 104] implemented the YOLOv3 DL algorithm for pothole detection with a mAP of 65.65% on 416 × 416 pixel images. The detected potholes were logged, and their locations translated for visualization on Google Maps using Google API.

3. Future Research Directions

Endeavors have been made to bridge the gaps and successfully implement DL algorithms in RCM using data acquired from sensors. In addition to the breakthrough results attained by researchers in this field, there exist several challenges that are listed below:

Quality and quantity are often considered as the keys to a balanced dataset and contribute to high performance in DL models. However, in spite of data augmentation techniques resulting in large pavement datasets [ 91], the performance of different DL models is not satisfactory. Thus arises a need to quantify the size of multi-class samples in a dataset and identify appropriate augmentation algorithms and methods to ensure a balanced, all-encompassing dataset.

Standardization of distress types provides a comprehensive evaluation platform to identify the best approach. However, it is observed that there exists no standard nomenclature for pavement distress annotations in the publicly available datasets, resulting in an ineffectual pavement distress analogy.

DL models demand large storage space. The size of high-performing models and their computational requirements pose a challenge to their usability in real-time scenarios.

To streamline the process above and beyond the existing challenges, the following directive actions need to be brought into effect.

(i) Real-time processing: on-board processing on UAVs and other platforms demand a lightweight though efficient detection model for the real-time assessment of pavements. There is a need to develop computationally reasonable and cost-effective models that can be deployed on sensor platforms to provide prompt results and avoid processing delays.

(ii) Data standardization: with large amounts of work being conducted in the development of a pavement management system, there is a need to standardize the data requirements to benefit cumulatively from the contributions made by the research community in the field. Data regulations can ensure a global collaboration towards a common goal, resulting in a robust solution to the problem.

(iii) Characterization of distresses: the autonomous detection of distresses has paved the way beyond the limitations faced by conventional systems. However, a step further in this direction that quantifies the extent of damage can help agencies to prioritize the regions demanding immediate attention. Thus, the physical characterization of distresses can streamline the evaluation system by working towards a cohesive solution.

(iv) Crowdsourcing platform: with data no longer being a limitation and ease of access to technology, a crowdsourcing platform can be developed for reporting distresses. This will add to the collective database and a robust system that can recognize and report the different distress types.

(v) Multidisciplinary research: civil engineers, computer scientists and data analysts can come together by contributing their expert domain knowledge to build an exhaustive distress evaluation system. A proper understanding of the pavement structures and their distresses can contribute to high-quality data resources and help in designing high-performance DL models.

(vi) High computing resources: DL models are data hungry and need high-performance computing resources. Cloud Computing and Edge Computing resources, including Internet-of-Things sensors, 5G and 6G, can be instantly provisioned with the ability to scale up and down. This will also allow systematic computing on a long-term basis devoid of memory and compute resource limitations.

(vii) Integrated system: innovation in system integration is another area on which research needs to be focused. The end-users are mostly government agencies and can benefit more from a package that delivers an end-to-end solution.

4. Conclusions

In this paper, the existing research work in RCM using next-generation sensors and AI methodologies was extensively reviewed and compared. The existing approaches are evaluated based on the data acquisition platforms under various AI approaches, especially DL for classification, segmentation and object detection. Data acquisition systems, which are a combination of non-intrusive sensors and their platforms, are at the center of the RCM system and involve the collection of 1D data, 2D visual data or 3D depth data. Every platform has its own advantages and limitations and complements each other's usability. For the tasks involving a simple classification of distresses, RGB sensors are a good option, while for a detailed study involving the various characteristics of distresses, LiDAR, laser, thermal or GPR sensors can be used. It is also necessary to appreciate the value and importance of data, along with the need to understand with clarity the definition of data. Whether the problem deals with classification, detection, or localization in RCM, at the core of every DL vision algorithm is a large collection of labeled images. Thus, another important step is the data annotation of collected data as ground truths, a crucial step in determining the accuracy of any DL model.

Developing DL models that accurately identify and characterize pavement distress is a challenging task. Overall, the CNN-based DL classification models show very high accuracy in detecting the presence or absence of multiple pavement distresses, and are better than conventional computer vision approaches in terms of performance; however, they fail to specify the location of distresses and thus lose their applicability in real-time scenarios. Over the course of time, CNN algorithms involving intelligent techniques to extract pavement distress features, such as transfer learning, pre-pooling layers, parallel CNNs, multi-stage CNNs, multi-source sensors, and IPTs integrated with CNNs, have been adopted to improve the existing performance of the DL classifiers.

Segmentation, which performs pixel-level classification employing CNNs, is another approach used in pavement distress analysis and can be used to extract the morphological characteristics of the distresses. However, not much work has been conducted in retrieving the physical characteristics of pavement damages, attributed to noisy post segmentation. Different approaches, such as connectivity maps and GANs, are being used to enhance segmentation outputs. Extensive work is being conducted in this area to precisely measure distress dimensions in terms of pixels; however, measurement accuracy, a decisive factor in determining the distress severity, is a contentious issue faced by many researchers.

Distress detection that involves classification and localization shows good results with 2D data and promises detailed analysis with 3D data, but not much work related to the measurements of distresses has been undertaken in this area. The detection of pavement damages helps in localizing the defects and has led many researchers to overlay this information on maps for navigation purposes. Innovative methods, such as thermal imaging, point cloud data, stereo-vision analysis, location-aware CNNs, mobile applications and UAV-based models for real-time assessment, to name a few, have been applied by many researchers to enhance the performance of existing CNN models. It has also been observed that CNN models have achieved a high performance in localizing distresses as a single class, when compared to localizing distresses under various classes. The performance of DL algorithms with advanced contributions from the scientific world and standardizations will continue to validate the use of DL approaches in the field of RCM. CNNs are currently the most reliable method to monitor the pavements for distresses, and the further integration with innovative technologies, affordable sensors and platforms shall encompass the futuristic, fully automated RCM systems.

Acknowledgments

The authors would like to thank the MITACS-Shastri Indo Canadian Institute (MITACS-SICI) partnership for providing financial support to conduct this research through the Globalink Research Award. This research is also supported by the Ontario Ministry of Colleges and Universities' Early Researcher Award funding provided to the corresponding author.

Abbreviations

AI

Artificial Intelligence

DCNN

Deep Convolutional Neural Network

DL

Deep Learning

FCN

Fully Convolutional Network

FOV

Field of View

GAN

Generative Adversarial Network

GPR

Ground Penetrating Radar

GPS

Global Positioning System

IoU

Intersection over Union

IPT

Image Processing Technique

IRI

International Roughness Index

LiDAR

Light Detection and Ranging

mAP

Mean Average Precision

ML

Machine Learning

RCM

Road Condition Monitoring

RDD

Road Damage Dataset

R-CNN

Region Convolutional Neural Network

RF

Random Forest

SVM

Support Vector Machine

UAV

Unmanned Aerial vehicle

YOLO

You Only Look Once

Author Contributions

Conceptualization, A.S. and E.R.; investigation, A.S. and E.R.; writing-original draft preparation, E.R.; writing-review and editing, A.S. and K.J.; supervision, A.S. and K.J.; funding acquisition, A.S., K.J. and E.R. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by MITACS-Shastri Indo Canadian Institute (MITACS-SICI) partnership and Ontario Ministry of Colleges and Universities' Early Researcher Award.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Table 6 provides the link to all the datasets that are publicly available.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Publisher's Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

4. Dunphy K., Sadhu A. Recent Developments in Structural Health Monitoring and Assessment-Opportunities and Challenges. World Scientific; Singapore: 2022. [ Google Scholar]

10. Vapnik V. Statistical Learning Theory. Wiley; New York, NY, USA: 1998. [ Google Scholar]

22. Christodoulou S., Kyriakou C., Hadjidemetriou G. Advanced Computing Strategies for Engineering. Elsevier; Amsterdam, The Netherlands: 2019. Pavement Patch Defects Detection and Classification Using Smartphones, Vibration Signals and Video Images; pp. 365-380. [ Google Scholar]

26. Shohel Rana D. Vibration Based Smart Pavement Monitoring System Using Vehicle Dynamics and Smartphone. enDAQ; Woburn, MA, USA: 2020. [ Google Scholar]

44. Fan R., Ozgunalp U., Hosking B., Liu M., Pitas I. Pothole Detection Based on Disparity Transformation and Road Surface Modeling. IEEE Trans. Image Process. 2020;29:897-908. doi: 10.1109/TIP.2019.2933750. [[ PubMed] [ CrossRef] Google Scholar]

46. Kawamura H., Nagata Y., Tokuno T., Ishida T., Mizutani T., Yamashita J. RILEM Bookseries. Volume 27. Springer Science and Business Media B.V.; Berlin/Heidelberg, Germany: 2022. Pavement Evaluation Method Using MMS; pp. 1455-1463. [[ CrossRef] Google Scholar]

56. Thomas A., Harikrishnan P.M., Nisha J.S., Gopi V.P., Palanisamy P. Advances in Intelligent Systems and Computing. Volume 1245. Springer; Singapore: 2021. Pothole and Speed Bump Classification Using a Five-Layer Simple Convolutional Neural Network; pp. 491-499. [[ CrossRef] Google Scholar]

64. Yang F., Zhang L., Yu S., Prokhorov D., Mei X., Ling H. IEEE Transactions on Intelligent Transportation Systems, under Review. 1 Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection. [(accessed on 26 March 2022)]. Available online: https://github.com/fyangneil/pavement-crack-detection

65. Gao Z., Peng B., Li T., Gou C. Generative Adversarial Networks for Road Crack Image Segmentation; Generative Adversarial Networks for Road Crack Image Segmentation. 2019. [(accessed on 26 March 2022)]. Available online: http://www.ieee.org/publications

66. Ronneberger O., Fischer P., Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. [(accessed on 1 May 2015)]. Available online: http://arxiv.org/abs/1505.04597

72. Ren S., He K., Girshick R., Sun J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. [(accessed on 26 March 2022)]. Available online: https://github.com/ [ PubMed]

107. Aravindkumar S., Varalakshmi P., Alagappan C. Artificial Intelligence and Technologies. Springer; Singapore: 2022. Automatic Road Surface Crack Detection Using Deep Learning Techniques; pp. 37-44. [[ CrossRef] Google Scholar]

Articles from Sensors (Basel, Switzerland) are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

# data[].attributes.attachment_link.title
Road Condition Monitoring Using Smart Sensing and Artificial Intelligence: A Review

# data[].attributes.attachment_link.description
Nicola Giaquinto, Academic Editor, Francesco Adamo, Academic Editor, and Maurizio Spadavecchia, Academic Editor Road condition monitoring (RCM) has been a demanding strategic research area in maintaining a large network of transport infrastructures. With advancements in computer vision and data mining techniques along with high computing resources, several innovative pavement distress evaluation systems have been developed in recent years.

# data[].attributes.body
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8125708/

# data[].attributes.attachment_link.title
Integrating Pavement Sensing Data for Pavement Condition Evaluation

# data[].attributes.attachment_link.description
Highway pavements are usually monitored in terms of their surface performance assessment, since the major cause that triggers maintenance is reduced pavement serviceability due to surface distresses, excessive pavement unevenness and/or texture loss.…

# data[].attributes.body
https://www.scopus.com/record/display.uri?eid=2-s2.0-84880149031&origin=resultslist&sort=plf-f&src=s&st1=active+geological+faults+on+highways&sid=2da198aca306c4c08d1cd44061d85340&sot=b&sdt=b&sl=51&s=TITLE-ABS-KEY%28active+geological+faults+on+highways%29&relpos=15&citeCnt=3&searchTerm=

# data[].attributes.body
https://www.scopus.com/record/display.uri?eid=2-s2.0-84984580977&origin=resultslist&sort=plf-f&src=s&st1=active+geological+faults+on+highways&sid=2da198aca306c4c08d1cd44061d85340&sot=b&sdt=b&sl=51&s=TITLE-ABS-KEY%28active+geological+faults+on+highways%29&relpos=10&citeCnt=3&searchTerm=

# data[].attributes.attachment_link.title
Scopus preview - Scopus - Welcome to Scopus

# data[].attributes.attachment_link.description
Elsevier’s Scopus, the largest abstract and citation database of peer-reviewed literature. Search and access research from the science, technology, medicine, social sciences and arts and humanities fields.

# data[].attributes.body
https://www.scopus.com/record/display.uri?eid=2-s2.0-85066129643&origin=resultslist&sort=plf-f&src=s&st1=active+geological+faults+on+highways&sid=2da198aca306c4c08d1cd44061d85340&sot=b&sdt=b&sl=51&s=TITLE-ABS-KEY%28active+geological+faults+on+highways%29&relpos=6&citeCnt=1&searchTerm=

# data[].attributes.body
https://www.canva.com/design/DAFiGAKvAx0/FVUaXfCE9ThbCW04ryjDNw/edit?utm_content=DAFiGAKvAx0&utm_campaign=designshare&utm_medium=link2&utm_source=sharebutton

# data[].attributes.body
http://repositorio.uancv.edu.pe/handle/UANCV/2601

# data[].attributes.body
https://www.redalyc.org/articulo.oa?id=56843309
https://www.redalyc.org/articulo.oa?id=32754216006
https://www.redalyc.org/articulo.oa?id=50530303

# data[].attributes.attachment_link.title
Neotectonic related geological risk at dams in the Mexico Basin: Guadalupe dam

# data[].attributes.attachment_link.description
Se analizaron los tres modelos tectónicos existentes del área de la Cuenca de México, con base en los cuales se elaboró un modelo tectónico que incorpora las estructuras geológicas comunes. Posteriormente se identificaron diversas obras civiles a lo largo de estas estructuras que pudieran sufrir efectos negativos para su seguro funcionamiento, debido a la actividad neotectónica de dichas estructuras.

# data[].attributes.body
http://www.elaguapotable.com/Folleto%20agua%20potable%20Degremont.pdf

# data[].attributes.headline
monitoreo del refuerzo geosintetico en actividades mineras

# data[].attributes.subject
monitoreo del refuerzo geosintetico en actividades mineras

# data[].attributes.body
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8037209/

# data[].attributes.attachment_link.readable_text
Evangelos J. Sapountzakis, Academic Editor

Abstract

This paper presents implementation of purpose-designed optical fibre Bragg grating (FBG) sensors intended for the monitoring of real values of strain in reinforced road structures in areas of mining activity. Two field test stations are described. The first enables analysis of the geogrid on concrete and ground subgrades. The second models the situation of subsoil deformation due to mining activity at different external loads. The paper presents a system of optical fibre sensors of strain and temperature dedicated for the investigated mattress. Laboratory tests were performed to determine the strain characteristic of the FBG sensor-geogrid system with respect to standard load. As a result, it was possible to establish the dependence of the geogrid strain on the forces occurring in it. This may be the basis for the analysis of the mining activity effect on right-of-way structures during precise strain measurements of a geogrid using FBG sensors embedded in it. The analysis of the results of measurements in the aspect of forecasted and actual static and dynamic effects of mining on the stability of a reinforced road structure is of key importance for detailed management of the road investment and for appropriate repair and modernization management of the road structure.

Keywords: monitoring fibre Bragg grating, mining areas, strain/stress distribution

1. Introduction

The analysis of the use of systems for long-term monitoring of the state of a road structure attracts a lot of attention, especially if the construction investment is located in an area of III geotechnical category with impacts on mining activity. By providing precise real-time information on the structure and its surroundings, it is possible to evaluate the state of the structure using diagnostic and prognostic tools based on the available data. The separation of intrinsic features from comprehensive monitoring data would be the key task in the assessment of the structure state and, consequently, an effective verification of the assumed unfavourable effects of mining activity compared to real impacts. The reasons for the use of components with optical fibres as sensor elements in the monitoring of the state of structures in use are well known.

The ever-growing demand for new linear investments usually involves improvement to land with difficult geotechnical conditions. Such land is often described as low-bearing soils, including the herein analysed soils in areas affected by mining impacts [ 1, 2]. The basic mechanisms determining the engineering methods of computational analysis of subsidence are connected with phenomena such as compaction, consolidation, changes in water conditions, or disturbances to the balance of soil massif. The road structure monitoring investigated in the paper is essential during the use of a construction investment to reduce factors generating investment risk.

Structures located in mining areas are subject to additional impacts due to surface deformation or subgrade/subsoil vibration. These impacts are transferred from the subsoil to the structure, which results in the occurrence of strains and internal forces in the structure. The monitoring of the strains provides current insight into the structure operation and creates the possibility of current checks on individual limit states of the structure.

In the case of an earthen structure, the strain-related impact of mining affects a significant portion of its area. In the zone of tensile strains, the subsoil and the embankment are loosened, and large limit-state areas are created. This leads to uneven settlement of the embankment crown, local subsidence, and horizontal displacement of the road pavement, as well as to local slides of slopes and pavement damage posing a hazard to safety. The presented monitoring system for a selected road structure scheme offers completely new possibilities for the use and protection, as well as renovation and modernization of road structures. In areas with mining impacts, the road structure subgrade is reinforced to increase the load-bearing capacity, reduce the subsidence of structures, prevent the loss of stability in the form of slips or landslides, and prevent the subgrade from liquefaction. Stabilization of the subgrade structure and mitigation of the effects of mining-related deformation were observed during numerous tests.

While choosing the method of subgrade reinforcement, the land development features should also be taken into account because the application of some methods, e.g., dynamic consolidation, may have a negative impact on existing building structures (resulting in damage to buildings due to transmission of vibrations from the subsoil or changes in hydrogeological conditions)-this is another thing that confirms the necessity and usefulness of monitoring in this field.

The data collected from the conducted analyses of selected models of the reinforcement of subgrades affected by the impact of mining activity additionally create the approach to the use of anthropogenic ground as a material for the proposed geosynthetic mattresses. This type of ground has already been tested in terms of its physical and strength parameters, as well as in the environmental aspect-utilization of materials from postmining waste dumps [ 3]. A database is being created for the environmental analysis of the issue.

Monitoring is an interdisciplinary issue. Due to its usefulness, it has a wide range of applications in the analysis of direct and indirect variables affecting variables related to the indicated aspects: securing the interests of the investor and the contractor of a construction investment, investment risk, especially in areas with the impact of mining activity, optimal assurance of the safety of investment implementation and use, and current assessment of the technical condition of a linear structure during its operation, as well as control and the possibility of getting an early warning of deformation of engineering structures in areas with an adverse mining impact. The presented empirical approaches to the monitoring of the interaction between the subsoil and the road structure provide a great deal of data that can be properly managed in a modern approach to the structure repair and modernization management. The monitoring data can generate appropriate models of the support of the decision-making process with methods suitable for the above-specified description of differentiating factors from the perspective of the researcher and analyst. The monitoring type and methodology and the range of the obtained data continuity for the time variable generate whole sections of statistics which can be used to find the answer to a given design-, strength-, or usability-related question, as well as economic, operating, or environmental problems. In this publication, the obtained data are analysed in the aspect of the road structure protection against the mining impact using a geomattress [ 4, 5]. For the set of data divided into main structural, protection, and reinforcement elements and monitoring measurement locations, a data-mining analysis was conducted to explore large data resources in search of systematic correlations between variables and then to evaluate the results by applying the detected patterns to new subsets of data.

The final goal of the data mining operations on the data coming from the created monitoring system is to predict the behaviour of the road structure and provide timely feedback to prevent the possibility of increasing the probability of the road structure damage. Predictive data mining gives direct economic benefits in the management of road use and renovation. The data from individual monitoring models can be used to build a model for specific patterns, together with evaluation and verification to obtain predicted values or classifications.

Many various monitoring technologies have been developed, including the sensor based on the optical fibre Bragg grating (FBG). This solution is now being investigated in terms of its possible application for road structures, herein for example in the area affected by mining activity. In 1966, the use of optical fibres for digital transmission of data was proposed by Kao, who later won the Nobel Prize for Physics in 2009.

This paper presents the results of testing the use of FBG sensors for the monitoring of the reinforced surface of a road structure affected by unfavourable static and dynamic impacts due to mining activity. The aim of the investigated monitoring system is to develop guidelines for the analysed variable that affect the comprehensive manner of utilization, operation, and management of a road structure in areas affected by tremors and subsidence due to mining.

This type of monitoring was recently investigated in [ 6, 7, 8, 9, 10, 11] and [ 12]; proposals for the use of the results of already tested monitoring systems to analyse the mining activity impact on residential and commercial buildings and road structures are signalled in [ 13, 14] or [ 15]. The strain measurement method based on the change in the wavelength on the Bragg grating is described in [ 2, 8, 16, 17, 18, 19].

As it is known, the Bragg grating is defined as periodic disturbance of the effective absorption coefficient and/or the optical fibre refractive index. Based on that, quasi-distributed systems are created for the detection of structural hazards. Herein, it concerns a road structure. The operation of mines, both past and present as well as future, generates imbalance of the rock mass. On the surface of the area, this results in direct effects, both continuous and discontinuous, indirect effects, and dynamic effects. All these effects have an unfavourable impact on buildings and road structures.

The main aim of the work is to use innovative FBG sensors to determine real values of strains and forces occurring in the geomattress depending on different service loads.

2. Presentation of the Problem for the Monitoring System

The aim of the testing and analyses presented herein is to demonstrate an original approach to the monitoring of changes in the stress-and-strain state of the mining subsoil for the reinforcement subsoil layer directly interacting with the construction of the road in areas with mining impact. The development of the civil engineering sector is directly linked to the development of the technology generating new solutions for the phases of design, realization, and operation of a construction investment.

The scope of innovations for the monitoring system creates new data and new schemes of analysis, e.g., for the investment risk calculation in the management of a road structure in an area affected by mining damage. Due to a continuous measurement of specific physical quantities, it is possible to control the behaviour of a civil engineering structure [ 4, 5, 15, 20, 21]. The authors point to the capabilities of a system that:

(i) identifies the actual behaviour of the subsoil under designed constant and variable loads as well as under the forecasted and unpredictable impact of mining activity;

(ii) provides data for online description of the subsoil-reinforcement element-road structure interaction for the analysed period of utilization and operation of a civil engineering structure;

(iii) is an important variable in the optimization of schedules of the structure repair and modernization and in the checking of the investment risk value depending on the variable related to the mining activity effect.

The guidelines for road design and construction binding in the EU require that earthen structures as well as surfaces should be designed and constructed so that any potential impacts and influences occurring during construction and use, including the effects of mining activity, should be carried appropriately. In these specific conditions, the structures should display adequate durability, taking account of the predicted service life, and should not succumb to destruction to an extent disproportionate to the cause. Meeting these requirements is equivalent to the need to ensure conditions in which load or usability limits are not exceeded not only in each individual element but also in the entire earthen structure together with the road surface.

The nature of damage to the road infrastructure in mining areas and of the mobilization of limit states of load capacity is of a completely different origin compared to other areas [ 1, 4, 22, 23, 24] and is the effect mainly of the susceptibility of earthen structures and surfaces to horizontal unit strain with a loosening character (ε (mm/m)). This problem is illustrated comprehensively in the diagram in Table 1 and Figure 1 , Figure 2 and Figure 3 .

Table 1

Mining Area Category

Slope
T (mm/m)

Curvature Radius
R (km)

Horizontal Strain
ε (mm/m)

At present, the majority of right-of-way structures in the road infrastructure still do not demonstrate sufficient structural resistance to such destructive impacts, which is the main cause of their damage. As stipulated by the regulations now in force, areas affected by mining activity should be protected according to the category of the mining area. The current classification of mining areas is presented, taking account of continuous and discontinuous deformation, paraseismic impacts and postmining areas. The measure of the hazard posed in a mining area by dynamic impacts both to newly erected and already existing structures is the assignment of a specific seismic zone to the area [ 25, 26] and [ 27, 28]. The seismic zone is described by parameters of the maximum ground vibrations that can occur in the area: acceleration and speed, as well as the subsoil design acceleration, all of which have to be taken into account in the design of civil structures in the area.

The indirect impacts affecting structures and related to the drainage of tertiary layers in the form of a large-size drainage basin are described using a single parameter: the mining area subsidence due to drainage. In areas where the impacts are not related to ground subsidence caused by mining, they do not result in mining damage to the development of the land or mining damage of a hydrogeological nature that is attributed in particular to changes in the stress-and-strain state of the ground. The definition of the mining area is directly connected to the range of the road structure interaction with the subsoil subjected to strain. According to ref. [ 29], this is the mass rock layer close to the surface, which is usually built of different soils, where the impact of the structure on the stresses and strains arising in the layer are considered as essential. The analysis of the results presented herein is based on the method of the Budryk-Knothe geometric-integral theory used in the majority of geomechanics in Silesia for prognostic analyses. It is the basic method of identification of kinematic limit states of investigated structures.

The analysed issues concern mining areas with subsoils reinforced with a geomattress to protect the right-of-way structure against deformation due to mining activity. The measure of the hazard posed to a mining area by impacts causing deformation is the mining area category. The category is described by indications of deformation. It is a variable that substantially affects the risk factor of the safe use of the structure. The basic indicators are subsidence-w, horizontal strain-ε (mm/m), land slope-T (mm/m), and curvature radius-R (km).

3. Experimental Setup

3.1. Test Stand Modelling

Two test stands were designed and made to achieve the research goals. The stands are equipped with optical FBG sensors of strain and displacement enabling a continuous measurement of strain in selected points of the geogrid. Test stand 1 is composed of the following layers: a 15 cm sand bed and the load-bearing mattress made of a geosynthetic grid filled with 16-32 mm aggregate with the thickness of 50 cm, compacted in layers every 25 cm. It is based partially on subsoil and partially on a reinforced concrete slab, which allows a comparison of strains in the two parts. Each subgrade has a separate FBG strain sensor embedded in the geogrid. The diagram of the test stand is shown in Figure 4 .

Test stand 2 enables an analysis of the case when there is no subgrade under the geogrid in two strips. It is composed of the following layers: geotextile, 800 × 1200 wooden pallets, and a load-bearing mattress filled with 31.5-63 mm aggregate with FBG sensors and 31.5-63 mm aggregate. The diagram of the test stand is shown in Figure 5 .

Two types of FBG sensors are applied: the strain sensor and the displacement sensor. Due to high values of the mattress deflection angles, the strain sensor proved useless and was no longer used in further stages of the research. On a single measuring line, the strain sensor may have 10 sensors with the wavelength difference of 5 nm with temperature compensation. A special technology of gluing the sensor into the geogrid in the middle of its span was developed. The sensor and its parameters are presented in Figure 6 .

The strain of the optical fibre sensor is the function of the wavelength (1) measured by the optical FBG-800 interrogator enabling dynamic measurements. The sampling frequency is 2000 Hz,

where A and B are constant Table 2 .

Table 2

Measurand

Description

A (με−1)

B (°C−1)

Δε (με)

Strain shift

7.7584 × 10 −07

5.8929 × 10 −06

λ 0,inst,strain (nm)

Initial strain wavelength

T 0.inst (°C)

Initial temperature

T act (°C)

Actual temperature

λ act,strain (nm)

Actual strain wavelength

3.2. Plan of Investigated Experiments

The testing was divided into two main parts: the laboratory stage and in situ testing. The laboratory stage included the process of calibration of the geogrid strains with the forces occurring in the grid. The in situ testing was carried out both during the construction of the test stand and during the load tests. The test stand preparation was divided into individual stages, which had a considerable impact on the geogrid strains Table 3 .

Table 3

Stage

Scope of Works Leading to the Construction
of the Test Stand

I

laying out the geogrid-no load

II

commencement of backfilling

III

mattress formation-first aggregate layer

IV

mattress formation-wrapping up the geogrid

V

mattress formation-second aggregate layer

VI

completion of the test stand construction

VII

tampering

The results of the geogrid strain-state testing during the construction of test stand 2 are presented in Figure 7 . The first stage of laying out the geogrid produced strains at the level of about 135 μstrain. The jump in the graph at 14:04:47 h to the value of 600 μstrain proves that the geogrid backfilling process began. Backfilling was performed by hand using shovels. It lasted about 15 min, which can be seen in the graph to 14:19:00 h. The next stage was the geogrid forming and wrapping up to shape the geomattress. During that time, the geogrid strains were constant and totalled about 750 μstrain. The final stage of the test stand preparation was filling the geogrid mattress with aggregate, which can be seen in the graph between 14:32 and 14:43 h. The works were finished at 14:44 h, which can be observed in the graph in the form of a stop to the increase in strains. Strain values then stay at the level of 1050 μstrain. This is the reference level for further load tests. In the initial stage an analysis was also conducted of strains arising during the aggregate compaction with a compactor. The aggregate was compacted separately over each type of subgrade (subsoil, concrete slab).

Distinct jumps in strains can be observed in Figure 8 when the compactor was directly over the sensor. After the compactor moved beyond the sensor, the strain value decreased at once. The graph also indicates that the geogrid mean strain rose cyclically. Characteristic steps (red line) appeared due to subsequent stages of the stand compaction, from 12 to 21 μstrain. Compaction of the aggregate over the part located on the natural soil resulted in different values of the geogrid strains. Distinct jumps in strains can be seen in Figure 9 when the compactor was directly over the sensor.

After the compactor moved to other places, the strain value in the tested location decreased considerably. After each compaction cycle, the geogrid suffered no plastic strain, which is illustrated by the red line. The natural soil thus plays the role of an elastic subgrade.

4. Results

The following results were obtained for individual stages of the testing for the designed monitoring technology. The technology was suited to the initial data of the investigated issue concerning the analysis of un-forecasted mining activity impacts on right-of-way road structures-roads with appropriate reinforcement of the subgrade using a geosynthetic mattress.

4.1. In Situ Testing Results

In situ tests were performed including: (a) static loading of the area under the FBG monitoring, (b) dynamic loading of the tested structure, (c) loading due to the use of construction equipment-88kN ascent of an excavator, and (d) putting the excavator supports on the area of the FBG monitoring. The second experiment performed on the test stand was a static test consisting in loading the structure with concrete slabs. Standard 50 × 50 × 7 cm paving slabs, each producing a 350 N load, were used for this purpose. The load was applied in layers with four slabs each. The weight of each of the layers produced a 1.4 kN load. The load was applied in places where the FBG sensor was installed. Moreover, in places under which there was a layer of pallets, an additional load was applied to additionally ballast the pallets to make the forces from the proper load to be taken over by the geogrid mainly. This additional load was not included in the weight of the layers producing the proper load. The experiment was carried out in stages by laying subsequent layers of slabs onto the stand. In the final stage of the testing, there were five layers of slabs on the stand. The total weight of the load was then 7.0 kN. The testing results are presented in Figure 10 .

For one layer of slabs-the weight equal to 1.4 kN-the strain values totalled about 70 μstrain. After the second layer was laid, when the weight rose to 2.8 kN, the strain level in the geogrid increased to 130 μstrain. The next layer of slabs caused a further rise in strain to the value of 240 μstrain. After the fourth layer of slabs was laid, the load totalled 5.6 kN, and the strain level rose to 380 μstrain. Lastly, the fifth layer of slabs resulted in another rise in strain to the value of 450 μstrain. The next experiment carried out on the field test stand was the free-fall test of concrete slabs simulating a dynamic load of the type of the Heaviside step function. Next, 50 × 50 × 7 cm concrete slabs were dropped from different heights of 30, 60, 90, and 120 cm, respectively. The idea of the testing is presented in Figure 11 , and the testing results are shown in Figure 12 .

The free fall of the slabs from different heights caused jump changes in the strain value. At a free fall from the height of 30 and 60 cm, the slab caused a strain jump to the value of 600 and 1000 μstrain, respectively. The slab free fall from 90 and 120 cm caused a jump in the geogrid strain value to 1400 and 2050 μstrain, respectively. This means that the impact of falling slabs caused a jump change in the values of the geogrid strains. The geogrid then returned to the initial state. In the graph it can also be noticed that after the slab free fall from 90 cm the geogrid plastic strain occurred and the strain level being the result of the next impact totalled 100 μstrain. In addition, in this experiment with impulse loads, the FBG sensors performed very well. They made it possible to analyse the geomattress strain state and provided an answer to the question whether or not the mattress had been damaged and whether or not the loads had produced plastic strain values. The next experiment performed on the field test stand for the geogrid monitoring was the run of the CASE Super R backhoe loader. The machine mass totals 8850 kg. The backhoe loader ran onto and left the stand three times. The loader run was controlled to ensure that its rear wheel was directly over the sensor. The loader ran onto the stand to cause maximum strain of the geogrid. Figure 13 shows the geogrid strain testing results during the loader run onto the test stand.

During the first run of the loader the maximum strain value totalled 1200 μstrain. It then dropped to the level of −200 μstrain to reach during the next run the maximum value of about 1380 μstrain. During the break between the loader second and last run onto the test stand, strains fell to the value of −300 μstrain. The last run produced a jump in strain to the value of 4300 μstrain. After the loader left the stand, the strain values dropped again to the level of −370 μstrain. This is the response of the geomattress system to its considerable strain.

The last experiment on the test stand was also carried out using the CASE Super R backhoe loader. This time the excavator supports were spread out and raised the machine. The supports were placed over the sensor to produce the highest loads of the area covered by the FBG monitoring and obtain the highest strains of the geogrid. The strains arising when the construction machine was placed on the geogrid are presented in Figure 14 .

Raising the loader on the supports caused an increase in the geogrid strain. After the first raising the strain level totalled 1780 μstrain. When the loader left the stand, the strain values decreased almost to zero to reach the level of 1650 μstrain at the next raising. After the loader left the stand, the strain level dropped again to the value of 50 μstrain. In the last test of raising the loader, the obtained strain value totalled 1770 μstrain. The drop in strain below zero from 10:43:42 h proves that the loader left the stand.

4.2. Laboratory Testing Results

The aim of the laboratory testing was experimental determination of relations between strains and loads (axial forces) in a unidirectional grid. For this purpose, a series of tensile tests of specimens from the geosynthetic grid were performed. The reference to the standard force was the dynamometer of a strength-testing machine (HOUNSFIELSD TEST EQUIPMENT 0133 model H50KS, Shakopee, MN, USA with Accuracy Class 0.5 ( Figure 15 ). The geogrid was the same as the one used in the field stand structure. The rupture test was performed according to standard EN ISO 10319:2008 geosynthetics-wide-width tensile test. Thirty (30) specimens 200 mm wide and 200 mm long were prepared for the testing to ensure the nominal length of 100 mm according to 6.3.3 PN-EN ISO 10319:2008. As the geogrid is reinforced unidirectionally, the specimen was ruptured in the longitudinal direction only. Special holders-elastomeric inserts-were designed to hold the strength-testing machine, which ensured that the loading process was carried out correctly. Subsequent reference testing was carried out in the same manner, except that the geogrid was equipped with an optical FBG sensor permanently glued into the grid using special technology.

Such embedding of the sensor into the geogrid worked very well in the case of field testing. The FBG sensor was glued into a 620-mm-long geogrid specimen to ensure the nominal length of 500 mm. Rubber inserts were fixed at the ends of the specimen to ensure the best assembly. The inserts were joined to the geogrid using a cyanoacrylate-based adhesive. The results of calibration tests are presented in Figure 16 .

The obtained calibration chart makes it possible to determine the value of the load occurring in the geogrid based on the measurement of the grid strain using an FBG sensor.

5. Discussion

The paper presents an innovative method of measuring strains of the geosynthetic grid using optical FBG sensors. The strain measurement, together with determination of the forces acting in the geogrid, is a very important element of the monitoring of the condition of a geotechnical structure and areas affected by mining damage. The values of strains and loads occurring in the geogrid were determined experimentally. The stand construction and the backfilling with aggregate caused strains at the level of 1050 μstrain corresponding to the load of 220 N/m. This strain state was adopted as the initial one. The highest strain values were caused by dynamic loads in the form of the Heaviside impulse of 2050 μstrain comparable to the excavator ascent onto the mattress. The lowest strain values of 70 μstrain were recorded during aggregate compaction on natural soil. The optical FBG sensors belong to the family of the fibre-optic sensors (FOS), and they perform very well in difficult terrain and environmental conditions. Individual FBG sensors can be connected easily to obtain a comprehensive measurement system of the entire structure. The sensors can be used to measure the structure strains on the one hand and the temperature of the structure or the moisture level on the other. However, their most important task is the strain measurement as strains are the factor that has the most essential impact on the safety of the use of a structure.

A very important value of the work is determination of real values of the geogrid strains and real values of forces acting in the grid. This is important as it creates the possibility of the geogrid stress-and-strain state evaluation. The works published so far do not refer to the values of forces in the geogrid and do not enable the grid stress-and-strain state evaluation. Most of them concentrate on monitoring the condition of a few key geotechnical structures, including soil nail systems, slopes, and piles, without any reference to the issue of the stress-and-strain state in the analysed elements. Ref. [ 30] points to the possibility of using the results of the grid strain measurement as input or boundary data for a numerical experiment using the finite difference or the finite element method. This will be the subject of the authors' further work. Many works concentrate on the FBG sensor mounting method. Two solutions can be distinguished. The first consists in applying an optical fibre with Bragg gratings over an intermediate and properly secured element. This works in structures with low deformability and is not proposed here because it would generate too large a measurement error. The second solution is to embed the optical fibre sensor in the tested structure using an adhesive. A special gluing technique was used in the analysed geogrid to embed in it not only the FBG strain sensor but, significantly enough, also a telecommunication optical fibre to ensure data transmission and the sensor supply with light. Indicating directions of further development, a proposal is made in [ 29] to construct 3D sensors. The research results point to the possibility of constructing orthogonal strain-measuring systems, 90- and 120-degree strain rosettes, as well as 3D systems. Another issue raised in to-date publications is the cost of the FBG monitoring system [ 30]. The experimentally determined strain values [ 19, 31, 32] are comparable to the values occurring in the geogrid. Undoubtedly, monitoring systems based on optical FBG sensors are relatively expensive. Low-cost solutions are being sought. The authors tested a new generation of 1 Hz optical interrogators with FBG sensors operating at the wavelength of 800 nm. Such a solution is equivalent to sensors working at the wavelength of 1550 nm and, most importantly, is twice as cheap as classical solutions. Attention should also be drawn to the unprecedented possibility of integrating different fibre-optic sensors to perform parallel measurements of strains, the forces occurring in the geogrid and the current temperature and moisture content. Another advantage of measurements based on optical fibre sensors is their fatigue strength, long service life, and the multiplexing capacity.

6. Conclusions

Appropriately processed, the data obtained from the geogrid monitoring can become the database for the analysis of risk presented by the operation of a structure in cases of un-forecasted mining activity impacts. Known forces of the interaction along the mining-activity-reinforcement-system-road-structure scheme in the monitored time interval should considerably improve the management of road structures in areas affected by unfavourable impacts of mining activity. The probability defining the risk factor related to the use of a road structure is usually expressed as frequency, i.e., the number of expected events in a unit of time. For structures provided with appropriate monitoring and covered by permanent analysis of variable values qualifying the structure for specific mining area categories, the frequencies of unfavourable and random events may vary by many orders of magnitude for the analysed logarithmic scale. The categories of the consequences of on-line monitoring of the work of a road structure can also be expressed in many matrices of risk related to the safety of the structure. The assumed schemes of data analysis give theses with very good results. Subsequent planned testing of buildings and structures in areas with forecasted mining impacts can become the basis for the verification of both the adopted monitoring methodology and the system of analysis of obtained data in the aspect of risk related to the use of an investment and optimization of schedules with the scope, type, and frequency of the performed repair and modernization management of the structure. Moreover, data are collected all the time for cost analysis of such monitoring of the structure. For the preliminarily developed risk matrices for selected types of mining damage repair work taking account of probability and consequence classes, the initial results are very promising. The combination of optical FBG sensors with geosynthetic materials and their application in road structures make them undoubtedly one of the innovative methods of strain measurement, and the presented calibration methodology enables the assessment of loads occurring in the geogrid.

Summarizing the initial assumptions, the adopted models, and the obtained final results, together with a discussion of current research trends, the following main points can be mentioned:

1. The applied system of FBG sensors enables precise and continuous monitoring of strains and loads of the road structure reinforcement in the form of a geomattress which takes over the impacts from the mining activity areas.

2. Compared to the existing systems, the presented monitoring system is characterized by exceptional resistance to difficult environmental conditions.

3. In the future, the monitoring information package can become the basis for a system warning against mining hazards presented to existing engineering structures. Such information is now indispensable for statistical analyses of the ground behaviour in the interaction with an engineering structure.

The growing set of monitoring data should be analysed in all interdisciplinary issues that are connected with the discussed monitoring system intended for road structures in areas affected by the impact of mining activity. Datasets are being created for a detailed environmental, economic, and performance analysis of the investigated investment in the form of a monitoring system intended for a road structure reinforced with a geosynthetic material.

Author Contributions

Conceptualization, J.J.; methodology, J.J.; software, M.G.-L.; validation, J.J. investigation, M.G.-L. and J.J.; resources, J.J.; writing-original draft preparation, M.G.-L. and D.L.; writing-review and editing, J.J. and M.G.-L.; visualization, M.G.-L. and D.L.; supervision, J.J. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

No new data were created or analyzed in this study. Data sharing is not applicable to this article.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Publisher's Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

3. Gwozdz-Lason M. Odpady kopalniane jako wytrzymały grunt antropogeniczny wykorzystywany w nowoczesnych konstrukcjach geotechnicznych-The anthropogenic ground come from mining wastes used in modern geotechnical structures) Saf. Eng. Anthropog. Objects. 2018:3-9. [ Google Scholar]

5. Francken L., Clauwaert C. Characterization and structural assessment of bound materials for flexible road structures; Proceedings of the 6-th International Conference on Asphalt Pavements; Ann Arbor, MI, USA. 13-17 July 1987; pp. 130-140. [ Google Scholar]

8. Wang H.P., Liu W.Q., Zhou Z., Wang S.H., Li Y., Guo Z.W. The behavior of a novel raw material-encapsulated FBG sensor for pavement monitoring; Proceedings of the 2011 International Conference on Optical Instruments and Technology: Optical Sensors and Applications; Beijing, China. 6-9 November 2011. [ Google Scholar]

12. Kadela M. Systemy monitorowania obiektów liniowych na terenach górniczych (Line objects monitoring systems in mining areas); Proceedings of the Materiały konferencyjne XII Dni Miernictwa Górniczego i Ochrony Terenów Górniczych; Gliwice, Poland. 12-14 June 2013; pp. 163-172. [ Google Scholar]

14. Buddery P., Morton C., Scott D., Owen N. A continuous roof and floor monitoring systems for tailgate roadways; Proceedings of the 18th Coal Operators' Conference, Mining Engineering; Wollongong, Australia. 7-9 February 2018; pp. 72-81. [ Google Scholar]

15. Mokrosz R. Obiekty Budowlane na Terenach Górniczych. Inżynier Budownictwa; Katowice, Poland: 2015. [ Google Scholar]

23. Wytyczne Wzmacniania Podłoża Gruntowego w Budownictwie Drogowym (Guidelines for Subsoil Reinforcement in Road Construction) GDDP-IBDiM; Warsaw, Poland: 2002. [ Google Scholar]

24. Metody Stosowania Geosyntetyków do Budowy i Wzmocnienia Nawierzchni i Ziemnych Budowli Drogowych (Geo-Synthetics Application Methods for the Construction and Reinforcement of Surfaces and Earthen Road Structures) IBDiM; Warsaw, Poland: 2003. [ Google Scholar]

28. Gwozdz-Lason M. The cost-effective and geotechnic safely buildings on the areas with mine exploitation; Proceedings of the 17th International Multidisciplinary Scientific GeoConference SGEM 2017; Albena, Bulgaria. 29 June-5 July 2017; pp. 877-884. [[ CrossRef] Google Scholar]

29. Projektowanie Budynków na Terenach Górniczych. Wydawnictwo Instytutu Techniki Budowlanej; Warsaw, Poland: 2006. ITB 2006 Instrukcje, Wytyczne, Poradniki, 416/2006. [ Google Scholar]

Articles from Materials are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

# data[].attributes.attachment_link.title
FBG Strain Monitoring of a Road Structure Reinforced with a Geosynthetic Mattress in Cases of Subsoil Deformation in Mining Activity Areas

# data[].attributes.attachment_link.description
Evangelos J. Sapountzakis, Academic Editor This paper presents implementation of purpose-designed optical fibre Bragg grating (FBG) sensors intended for the monitoring of real values of strain in reinforced road structures in areas of mining activity. Two field test stations are described. The first enables analysis of the geogrid on concrete and ground subgrades.

# data[].attributes.body
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8399111/
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9781037/

# data[].attributes.attachment_link.title
Polymer Geogrids: A Review of Material, Design and Structure Relationships

# data[].attributes.attachment_link.description
Geogrids are a class of geosynthetic materials made of polymer materials with widespread transportation, infrastructure, and structural applications. Geogrids are now routinely used in soil stabilization applications ranging from reinforcing walls to…

# data[].attributes.headline
IMPLEMENTACION DE MATERIALES EN PAVIMENTO PARA MITIGAR DAÑO EN FALLAS GEOLOGICAS

# data[].attributes.subject
IMPLEMENTACION DE MATERIALES EN PAVIMENTO PARA MITIGAR DAÑO EN FALLAS GEOLOGICAS

# data[].attributes.body
https://repositorio.umsa.bo/handle/123456789/30649
https://repositorio.ucv.edu.pe/handle/20.500.12692/34667
https://scholar.google.es/scholar?hl=es&as_sdt=0%2C5&q=pavimentos+en+falla+geologica&btnG=

# data[].attributes.attachment_link.title
Pavimento articulado una alternativa de solución ante fallas geológicas en la construcción de infraestructura vial

# data[].attributes.attachment_link.description
Las obras viales son de vital importancia para el desarrollo integral de un país, bajo este criterio es fundamental contar con vías bien construidas y conservadas, siendo el único medio que posibilita el transporte de personas y cargas, sin embargo, se debe tener en cuenta que existen fenómenos naturales y ambientales que ocasionan deslizamientos, hundimientos, derrumbes, etc.

# data[].attributes.body
http://ve.scielo.org/scielo.php?script=sci_arttext&pid=S1316-48212008000100007&lang=es

# data[].attributes.attachment_link.readable_text
Inserny es Investigadora colaboradora con el Departamento de Electrónica de La Ing. Virginia la UNEXPO Vicerrectorado Puerto Ordaz, Venezuela. Edificio de Ingeniería Eléctrica, Final Calle China, Alta Vista Sur, Telf.-Fax 58-286-9621205, correo electrónico virginiainserny@msn.com .

El Ing. Amer Aboul es Especialista de Acceso en el Dpto. de Implementación en Movistar, Av. Guayana con Calle Cuchiveros, Alta Vista, Ciudad Guayana, Venezuela. Telf. 0414-8955088, e-mail: amer.aboulhosn.ittaca@telefonica.com.ve .

El Ing. José Cañas es Investigador colaborador en el Departamento de Electrónica de la UNEXPO Vicerrectorado Puerto Ordaz, Venezuela. teléfono y fax 58-286-9621205, correo electrónico cicunexpo@gmail.com .

El Dr. Ángel Custodio es Profesor Agregado en el Dpto. de Ingeniería Electrónica del Centro de Instrumentación y Control y Coordinador Regional de Postgrado de la misma Universidad Unexpo, Puerto Ordaz, Venezuela. mismo teléfono, fax y correo electrónico.

En la siguiente investigación se desarrolló una propuesta para la automatización de la planta de tratamiento de agua potable de C.V.G GOSH Toro Muerto, que permita aumentar la eficiencia en la producción de la planta y así brindar un mejor y más óptimo servicio a la comunidad de Puerto Ordaz. En los resultados de la investigación se muestran la elección de equipos PLC para la realización del sistema de control, así como también el cable de fibra óptica conectado en una topología tipo estrella para la comunicación de los dispositivos. Se propone además un SCADA desarrollado en Delphi como primer paso de un sistema basado en Software Libre desarrollado por la UNEXPO. La investigación se realizó mediante un estudio de tipo documental y de campo, al fundamentarse en la consulta de manuales, libros, documentos de Internet; y en la planta, teniendo que tomar medidas y muestras para conocer el funcionamiento de una planta de tratamiento de agua potable y poder compararla con el funcionamiento de una planta de tratamiento típica.

Palabras clave : Planta de Tratamiento de agua/ Sistema de supervisión centralizado/ CVG Gosh/ Ciudad Guayana.

En cuanto a Ciudad Guayana ubicada entre dos grandes ríos, el Orinoco y el Caroní, sin contar con las fuentes de aguas subterráneas, es una de las ciudades con mayor posibilidad de tener un suministro de agua potable continuo. Sin embargo, el crecimiento de la población ha generado un impacto en las plantas de tratamiento de agua existentes. Hoy en día el preciado líquido está siendo regulado, dando lugar a que distintas zonas de la región queden sin servicio de agua a determinadas horas. Esto hace necesario un aumento de la producción de agua por parte de las plantas de tratamiento, así como el óptimo funcionamiento de los equipos en sus distintas áreas.

Actualmente, en la Planta de Tratamiento de Agua Potable no existe un sistema de control automatizado para la supervisión y el monitoreo de las funciones de la planta, con el fin de asegurar su correcto funcionamiento y poder brindar información rápida y oportuna sobre las características, ubicación, estado de los componentes que conforman la red de agua potable, tanto a los usuarios como a la organización. Por estas razones se requiere un sistema de control de todas estas variables para lograr un correcto funcionamiento de la planta.

Con el diseño y posterior implementación de este sistema de supervisión y control se pretende mejorar la calidad del servicio que presta esta empresa a la comunidad de Puerto Ordaz. El presente trabajo tiene como objetivo desarrollar un sistema de instrumentación y control necesario para automatizar la Planta de tratamiento de Agua Potable de CVG GOSH Planta Toro Muerto.

Es proyectiva debido a que se intenta proponer una solución al problema que se tiene en la actualidad, diseñando un sistema de instrumentación y control que permita elevar la eficiencia de la Planta de Tratamiento de Agua de CVG GOSH Toro Muerto, basado en un estudio de tipo descriptivo. La obtención de la información o datos que posibilita conocer el sistema y su funcionamiento se realizó por medio de la observación de la dinámica de proceso, entrevistas y consulta de material escrito. No fue necesario el desarrollo de métodos estadísticos, ya que la propuesta de automatización está basada en el funcionamiento global del sistema, y por tanto, los resultados son obtenidos por pruebas empíricas.

En la presente investigación las unidades objeto de análisis son los diferentes equipos con los que cuenta la Empresa ubicados en las plantas de tratamiento, tales como: motores, bombas, tableros eléctricos, transformadores, etc.

El área de trabajo se encuentra ubicada en Toro Muerto consistente en una Planta de Tratamiento de Agua Potable de CVG GOSH dividida en tres etapas: captación local, tratamiento y su posterior distribución ( Figura 1).

El trabajo propone instalar un sistema supervisorio para controlar las distintas operaciones de potabilización en el Acueducto de Toro Muerto Puerto Ordaz, a partir de los procesos de captación, tratamiento y distribución, involucrando la acción de arranque, parada de equipos en modo local (manual y automático) y remota, con la supervisión de las variables asociadas a cada proceso: presión, caudal, temperatura, potencia, tensión y corriente ( Figura 2).

Con esta propuesta se mejora el funcionamiento de la planta al manejar rápidamente los caudales de producción, consumo de aditivos químicos, costo energético del agua, registros de eventos tales como las paradas. El sistema debe ser modificado con relativa facilidad en caso de futuras expansiones ya que se pretende utilizar para ello dispositivos programables modulares en una red que integre los procesos de potabilización ( Figura 3).

La estructura del programa implementado se muestra en 4. Los equipos son almacenados en bases de datos y archivados en grupos similares: motores, válvulas y tanques. Con el módulo de configuración de componentes se organiza el sistema en las diversas etapas del proceso de tratamiento: captación, tratamiento y distribución. El usuario tiene acceso a todas las opciones y su respectiva manipulación.

El sistema a instalar tiene dos modalidades de funcionamiento: un Modo Local y un Modo Remoto. El Modo Remoto se divide en dos partes, Control Manual y Control Automático. Para la selección de estos modos se cuenta con un selector ubicado en el panel del equipo localizado en el campo. Todo el sistema de automatización tiene la misma filosofía de control y está regido por el diagrama en bloques mostrado en 5.

En el caso de la Planta de Tratamiento Toro Muerto Puerto Ordaz de CVG GOSH, los equipos a supervisar y controlar no requieren de operaciones matemáticas complejas de las variables, ya que las bombas a utilizar no necesitan de control de velocidad, ni posición de válvulas pues sólo se arrancan y detienen, en algunos sectores, por ejemplo: PT-1, Red Media 1(RM-1), Red Media 2 (RM-2), Red Superior 1 (RS-1). En los casos de captación se requiere de las acciones de arranque y parada para efectos de mantenimiento y corrección de fallas, ya que estos equipos funcionan las 24 horas. En el caso de los dosificadores cuentan con sus propios sistemas de control y en el filtrado se pueden efectuar las operaciones de lavado con diagramas de tiempo, utilizando Contra - Flujo.

En 6 se muestra un plano de planta de la red propuesta diseñada.

a. Selección de controladores específicos para las áreas de potabilización y agregado de sustancias químicas. En se muestra la comparativa de selección.

d. Desarrollo de un software de control y supervisión basado en Delphi, que representa un primer paso para el desarrollo de software libre para el SCADA

1. Rivas Maritza, Rivero Diomar y otros. Sistema de Información de la Red de Agua Potable. Universidad Central de Venezuela (SRAP-UCV). Caracas, Venezuela. 2002, 248 pp. [ Links ]

2. Ricardo Barrios. Sistema portátil de adquisición de datos con comunicación tcp/ip. Trabajo de Grado. Unexpo Puerto Ordaz, Venezuela, 2005, 341 pp. Volumen 12, Nº 46, marzo 2008. 43-48. [ Links ]

# data[].attributes.attachment_link.title
Diseño de un sistema de supervisión centralizado para las plantas de tratamiento de agua de CVG GOSH. Ciudad Guayana

# data[].attributes.attachment_link.description
Inserny es Investigadora colaboradora con el Departamento de Electrónica de La Ing. Virginia la UNEXPO Vicerrectorado Puerto Ordaz, Venezuela. Edificio de Ingeniería Eléctrica, Final Calle China, Alta Vista Sur, Telf.-Fax 58-286-9621205, correo electrónico virginiainserny@msn.com . El Ing. Amer Aboul es Especialista de Acceso en el Dpto. de Implementación en Movistar, Av.

# data[].attributes.body
-diseño de pavimento para mitigar el daño en fallas geologicas.
-PTAP en zonas rurales innovar.

# data[].attributes.body
Implementar PTAP compacta de zona sur