Date of Award

3-2024

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Computer and Network Engineering

First Advisor

Dr. Hesham El-Sayed

Abstract

Autonomous driving has the potential to bring significant changes and benefits to various aspects of transportation. Autonomous vehicles (AVs) use a combination of advanced sensors, cameras, radar, lidar, GPS, maps, and AI algorithms to perceive their environment, make decisions, and control their movements. Though there is a significant increase in the AVs utility, there are several challenges associated with the AVs among which ensuring safety and security for a reliable drive is still an existing challenge. The majority of accidents involving the AVs result from faulty decision-making resulting in fatal incidents. Multiple elements contribute to the flawed decision-making in autonomous vehicles (AVs), with inaccurate context creation being highlighted as one of the pivotal factors. For comprehensive safety across diverse driving environments, an autonomous vehicle must adeptly and dependably interpret its surroundings. Inadequate data acquisition from diverse sources and insufficient data pre-processing are the primary factors contributing to this inaccurate formation of context. To enhance environmental awareness and boost decision-making precision in autonomous vehicles (AVs), a versatile framework has been proposed. This framework incorporates multiple modules designed to oversee vital tasks such as collecting and organizing sensory data in diverse formats, extracting pertinent features, fusing them effectively, establishing precise context, and creating inventive and rapid decision protocols for timely decision-making in AVs. This research introduces innovative mechanisms for sensory data classification and versatile machine learning (ML) models for feature extraction and data fusion. A novel mechanism is proposed for instant rule framing and decision-making based on the fused data. This endeavour is commenced by presenting an outline of the functionality of the proposed framework, specifically highlighting image and video data formats, which hold prominence in sensory data. Efficient models have been suggested with the aim of extracting vital image attributes, including edges, colour, height, and width. Furthermore, an ingenious mathematical model is introduced, utilizing advanced matrix transformations and progressive modes to convert two-dimensional image data formats into three-dimensional ones. This mathematical model serves as the fundamental kernel function for the proposed Convolutional Neural Network (CNN) model, enabling the fusion of different image data formats. Additional innovative concepts and mechanisms are introduced in this research to enhance the performance of the proposed models. The extension of the proposed edge detection model encompasses detecting edges in all directions within the input image, surpassing the previous limitation of solely identifying horizontal and vertical edges. Advanced mathematical models incorporating genetic mutation techniques are proposed to accomplish this task. Versatile kernel functions are developed to process 3D point cloud sensory data, which are integrated into the proposed Generative Adversarial Network (GAN) model for classifying and fusing different image data formats. The extended research effectively completes the functionalities of the remaining modules within the proposed framework. In the expanded work, novel models have been introduced to efficiently combine textual and audio data. Additionally, versatile models for object detection and classification have been presented, enhancing the accurate recognition and categorization of objects. Advanced techniques such as ensembling, gating, and filtering are incorporated to select the most suitable object detection and classification model. Further, innovative methodologies are proposed to establish accurate context and decision rules. The performance evaluation of the proposed models utilizes widely recognized datasets namely KITTI, nuScenes, RADIATE, OSU, BPEM and GeoTiles. The suggested image fusion model achieved an accuracy of 98% and demonstrated a faster execution time (0.98s) compared to other well-known image fusion models. Alternatively, the suggested object detection model demonstrated a sensitivity of 0.65 and an average precision of 0.85, confirming its improved performance than other widely acknowledged object detection models. Further results are discussed in the Experimental Analysis portion of Chapter 5, which is the outcome of extensive investigations carried out on several parameters to evaluate the suggested models.

Arabic Abstract


إطﺎر ﻋﻤﻞ ﻣﺴﺘﺤﺪث ﻓﻲ دﻣﺞ اﻟﺒﯿﺎﻧﺎت ﻟﺘﻄﻮﯾﺮ إطﺎر ﻣﺤﺘﻮى اﻟﻤﺮﻛﺒﺎت ذاﺗﯿﺔ اﻟﻘﯿﺎدة ﻣﻦ أﺟﻞ اﺗﺨﺎذ ﻗﺮارات دﻗﯿﻘﺔ

تتمتع اﻟﻘﯿﺎدة اﻟﺬاﺗﯿﺔ ﺑﺎﻟﻘﺪرة ﻋﻠﻰ إﺣﺪاث ﺗﻐﯿﯿﺮات وﻓﻮاﺋﺪ ﻛﺒﯿﺮة ﻓﻲ ﻣﺨﺘﻠﻒ ﺟﻮاﻧﺐ اﻟﻨﻘﻞ. وﺗﺴﺘﺨﺪم اﻟﻤﺮﻛﺒﺎت ذاﺗﯿﺔ اﻟﻘﯿﺎدة (AVs) ﻣﺠﻤﻮﻋﺔ ﻣﻦ أﺟﮭﺰة اﻻﺳﺘﺸﻌﺎر اﻟﻤﺘﻄﻮرة، واﻟﻜﺎﻣﯿﺮات، واﻟﺮادارات، و ﺗﻘﻨﯿﺔ Laider ، وﻧﻈﺎم ﺗﺤﺪﯾﺪ اﻟﻤﻮاﻗﻊ اﻟﻌﺎﻟﻢ (GPS) ، واﻟﺨﺮاﺋﻂ، وﺧﻮارزﻣﯿﺎت اﻟﺬﻛﺎء اﻻﺻﻄﻨﺎﻋﻲ ﻟﻠﺘﻌﺮف ﻋﻠﻰ ﺑﯿﺌﺘﮭﺎ، واﺗﺨﺎذ اﻟﻘﺮارات، واﻟﺘﺤﻜﻢ ﻓﻲ ﺣﺮﻛﺘﮭﺎ. ﻋﻠﻰ اﻟﺮﻏﻢ ﻣﻦ اﻻزدﯾﺎد ﻓﻲ اﺳﺘﺨﺪام اﻟﻤﺮﻛﺒﺎت ذاﺗﯿﺔ اﻟﻘﯿﺎدة إﻻ أﻧﮫ ﻣﺎزاﻟﺖ ھﻨﺎك اﻟﻌﺪﯾﺪ ﻣﻦ اﻟﺘﺤﺪﯾﺎت اﻟﻤﺮﺗﺒﻄﺔ ﺑﻀﻤﺎن اﻷﻣﻦ واﻟﺴﻼﻣﺔ وﺗﻮﻓﯿﺮ ﻗﯿﺎدة ﻣﻮﺛﻮﻗﺔ. إن أﻏﻠﺐ أﺳﺒﺎب ﺣﻮادث ﻣﺮﻛﺒﺎت ذاﺗﯿﺔ اﻟﻘﯿﺎدة ھﻲ اﺗﺨﺎذ ﻗﺮارات ﺧﺎطﺌﺔ ﻣﻤﺎ ﯾﺆدي إﻟﻰ ﺣﻮادث ﻣﻤﯿﺘﮫ. وﺗﺴﺎھﻢ ﻋﻮاﻣﻞ ﻣﺘﻌﺪدة ﻓﻲ اﺗﺨﺎذ اﻟﻘﺮار اﻟﺨﺎطﺊ ﻓﻲ اﻟﻤﺮﻛﺒﺎت ذاﺗﯿﺔ اﻟﻘﯿﺎدة، وﯾﻌﺘﺒﺮ وﺟﻮد ﻣﺤﺘﻮى ﻏﯿﺮ دﻗﯿﻖ أﺣﺪ اﻟﻌﻮاﻣﻞ اﻟﻤﺤﻮرﯾﺔ اﻟﺘﻲ ﯾﺴﻠﻂ ﻋﻠﯿﮭﺎ اﻟﻀﻮء. وﻟﺘﺤﻘﯿﻖ اﻟﺴﻼﻣﺔ اﻟﺸﺎﻣﻠﺔ ﻋﺒﺮ ﺑﯿﺌﺎت اﻟﻘﯿﺎدة اﻟﻤﺘﻨﻮﻋﺔ، ﯾﺠﺐ ﻋﻠﻰ اﻟﻤﺮﻛﺒﺔ ذاﺗﯿﺔ اﻟﻘﯿﺎدة ﺗﻔﺴﯿﺮ ﻣﺤﯿﻄﮭﺎ ﺑﻤﮭﺎرة وﻣﻮﺛﻮﻗﯿﺔ. وﯾﻌﺪ اﻛﺘﺴﺎب ﺑﯿﺎﻧﺎت ﻏﯿﺮ ﻛﺎﻓﯿﺔ ﻣﻦ ﻣﺼﺎدر ﻣﺘﺸﻌﺒﺔ إﺿﺎﻓﺔ إﻟﻰ ﻋﺪم ﻛﻔﺎﯾﺔ اﻟﻤﻌﺎﻟﺠﺔ اﻟﻤﺴﺒﻘﺔ ﻟﻠﺒﯿﺎﻧﺎت ﻣﻦ اﻟﻌﻮاﻣﻞ اﻷﺳﺎﺳﯿﺔ ﻟﺘﻜﻮﯾﻦ ﻣﺤﺘﻮى ﻏﯿﺮ دﻗﯿﻖ. ﺗﻢ اﻗﺘﺮاح إطﺎر ﻋﻤﻞ ﻣﺘﻌﺪد اﻻﺳﺘﺨﺪاﻣﺎت ﻟﺘﻄﻮﯾﺮ اﻟﻮﻋﻲ اﻟﺒﯿﺌﻲ ودﻗﺔ اﺗﺨﺎذ اﻟﻘﺮارات ﻓﻲ اﻟﻤﺮﻛﺒﺎت ذاﺗﯿﺔ اﻟﻘﯿﺎدة. ﯾﺸﺘﻤﻞ إطﺎر اﻟﻌﻤﻞ اﻟﻤﻘﺘﺮح ﻋﻠﻰ وﺣﺪات ﻣﺘﻌﺪدة ﻣ ﺼﻤﻤﺔ ﻟﻺﺷﺮاف ﻋﻠﻰ اﻟﻤﮭﺎم اﻟﻀﺮورﯾﺔ ﻛﺠﻤﻊ وﺗﻨﻈﯿﻢ اﻟﺒﯿﺎﻧﺎت اﻟﻨﺎﺗﺠﺔ ﻋﻦ أﺟﮭﺰة اﻻﺳﺘﺸﻌﺎر واﻟﺘﻲ ﺗﻢ ﺗﺠﻤﯿﻌﮭﺎ ﻣﻦ ﻣﺼﺎدر ﻣﺘﺸﻌﺒﺔ وﺑﺘﻨﺴﯿﻘﺎت ﻣﺘﻨﻮﻋﺔ، اﺳﺘﺨﺮاج اﻟﻤﯿﺰات ذات اﻟﺼﻠﺔ، ودﻣﺠﮭﺎ ﺑﺼﻮرة ﻓﻌﺎﻟﺔ، اﻧﺸﺎء ﻣﺤﺘﻮى دﻗﯿﻖ، وإﻧﺸﺎء ﺑﺮوﺗﻮﻛﻮﻻت ﻟﻠﻘﺮارات ﺗﺘﺼﻒ ﺑﺄﻧﮭﺎ ﻣﺒﺘﻜﺮة وﺳﺮﯾﻌﺔ وﺗﺴﺎھﻢ ﻓﻲ اﺗﺨﺎذ اﻟﻘﺮار ﻓﻲ اﻟﻮﻗﺖ اﻟﻤﻨﺎﺳﺐ. ﯾﻄﺮح ھﺬا اﻟﺒﺤﺚ آﻟﯿﺎت ﻣﺒﺘﻜﺮة ﻟﺘﺼﻨﯿﻒ اﻟﺒﯿﺎﻧﺎت اﻟﺤﺴﯿﺔ وﻧﻤﺎذج اﻟﺘﻌﻠﻢ اﻵﻟﻲ ﻣﺘﻌﺪدة اﻻﺳﺘﺨﺪاﻣﺎت ﻻﺳﺘﺨﺮاج اﻟﻤﯿﺰات ودﻣﺞ اﻟﺒﯿﺎﻧﺎت. وﺑﺪأ ھﺬا اﻟﻤﺠﮭﻮد ﺑﺘﻘﺪﯾﻢ ﻣﺨﻄﻂ ﺗﻔﺼﯿﻠﻲ ﻟﻮظﺎﺋﻒ إطﺎر اﻟﻌﻤﻞ اﻟﻤﻘﺘﺮح، وﻋﻠﻰ وﺟﮫ اﻟﺘﺤﺪﯾﺪ ﺗﻨﺴﯿﻘﺎت ﺑﯿﺎﻧﺎت اﻟﺼﻮرة واﻟﻔﯿﺪﯾﻮ واﻟﺘﻲ ﺗﺤﻈﻰ ﺑﺄھﻤﯿﺔ ﻛﺒﯿﺮة ﻓﻲ اﻟﺒﯿﺎﻧﺎت اﻟﺤﺴﯿﺔ. وﻗﺪ ﺗﻢ اﻗﺘﺮاح ﻧﻤﺎذج ﻓﻌﺎﻟﺔ ﺑﮭﺪف اﺳﺘﺨﺮاج ﺳﻤﺎت اﻟﺼﻮرة اﻷﺳﺎﺳﯿﺔ، ﺑﻤﺎ ﻓﻲ ذﻟﻚ اﻟﺤﻮاف واﻟﻠﻮن واﻻرﺗﻔﺎع واﻟﻌﺮض، إﺿﺎﻓﺔ إﻟﻰ ﻧﻤﻮذج ﺣﺴﺎﺑﻲ ﻣﺒﺘﻜﺮ ﯾُﺴﺘﺨﺪم ﻓﻲ ﺗﺤﻮﯾﻞ ﺑﯿﺎﻧﺎت اﻟﺼﻮر ﻓﻲ اﻟﻤﺼﻔﻮﻓﺎت اﻟﻤﺘﻄﻮرة ﻣﻦ ﻣﺼﻔﻮﻓﺎت ﺛﻨﺎﺋﯿﺔ اﻷﺑﻌﺎد إﻟﻰ ﻣﺼﻔﻮﻓﺎت ﺛﻼﺛﯿﺔ اﻷﺑﻌﺎد. ﯾﻘﻮم ھﺬا اﻟﻨﻤﻮذج اﻟﺤﺴﺎﺑﻲ ﺑﻌﻤﻞ وظﯿﻔﺔ اﻟﻨﻮاة اﻷﺳﺎﺳﯿﺔ ﻟﻨﻤﻮذج اﻟﺸﺒﻜﺔ اﻟﻌﺼﺒﯿﺔ اﻟﻤﻠﺘﻔﺔ اﻟﻤﻘﺘﺮح، ﻣﻤﺎ ﯾﺘﯿﺢ دﻣﺞ ﺗﻨﺴﯿﻘﺎت ﺑﯿﺎﻧﺎت اﻟﺼﻮر اﻟﻤﺨﺘﻠﻔﺔ. وﺗﻢ ﺗﻘﺪﯾﻢ ﻓﻲ ھﺬا اﻟﺒﺤﺚ ﻣﻔﺎھﯿﻢ وآﻟﯿﺎت إﺿﺎﻓﯿﺔ وﻣﺒﺘﻜﺮة ﻟﺘﻌﺰﯾﺰ أداء اﻟﻨﻤﺎذج اﻟﻤﻘﺘﺮﺣﺔ. وﯾﺘﺼﻒ ﻧﻤﻮذج اﻟﻜﺸﻒ ﺑﻤﯿﺰة إﺿﺎﻓﯿﺔ ﺗﺸﻤﻞ اﻟﻜﺸﻒ ﻋﻦ اﻟﺤﻮاف ﻓﻲ ﺟﻤﯿﻊ اﺗﺠﺎھﺎت اﻟﺼﻮرة اﻟﻤُﺪﺧﻠﺔ، ﻣﺘﺠﺎوزاً اﻟﻤﺤﺪودﯾﺔ ﻓﻲ اﻷﻧﻈﻤﺔ اﻟﻤﺴﺒﻘﺔ واﻟﻤﺘﻤﺜﻠﺔ ﻓﻲ ﺗﺤﺪﯾﺪ اﻟﺤﻮاف اﻷﻓﻘﯿﺔ واﻟﺮأﺳﯿﺔ ﻓﻘﻂ. اﻟﻨﻤﻮذج اﻟﺤﺴﺎﺑﻲ ﯾﺘﻀﻤﻦ ﺗﻘﻨﯿﺎت ﺗﺘﺼﻒ ﺑﺎﻟﻄﻔﺮة اﻟﺠﯿﻨﯿﺔ ﻷداء ھﺬه اﻟﻤﮭﻤﺔ. ﺗﻢ ﺗﻄﻮﯾﺮ وظﺎﺋﻒ اﻟﻨﻮ اة ﻣﺘﻌﺪدة اﻻﺳﺘﺨﺪاﻣﺎت ﻟﻤﻌﺎﻟﺠﺔ اﻟﺒﯿﺎﻧﺎت اﻟﺤﺴﯿﺔ اﻟﺴﺤﺎﺑﯿﺔ ﺛﻼﺛﯿﺔ اﻷﺑﻌﺎد، واﻟﺘﻲ ﺗﻢ دﻣﺠﮭﺎ ﻓﻲ ﻧﻤﻮذج اﻟﺸﺒﻜﺔ اﻟﺘﻮﻟﯿﺪﯾﺔ اﻟﺘﻨﺎﻓﺴﯿﺔ (GAN) اﻟﻤﻘﺘﺮح ﻟﺘﺼﻨﯿﻒ ودﻣﺞ ﺗﻨﺴﯿﻘﺎت ﺑﯿﺎﻧﺎت اﻟﺼﻮر اﻟﻤﺨﺘﻠﻔﺔ. وأﻛﻤﻞ اﻟﺒﺤﺚ وظﺎﺋﻒ اﻟﻮﺣﺪات اﻟﻤﺘﺒﻘﯿﺔ ﺿﻤﻦ اﻹطﺎر اﻟﻤﻘﺘﺮح ﺑﺼﻮرة ﻓﻌﺎﻟﺔ. وﻓﻲ ھﺬا اﻟﺒﺤﺚ اﻟﻤﻤﺘﺪ ﻟﻸﺑﺤﺎث اﻟﺴﺎﺑﻘﺔ ﺗﻢ ﺗﻘﺪﯾﻢ ﻧﻤﻮذج ﻣﺒﺘﻜﺮ ﻟﺪﻣﺞ اﻟﺒﯿﺎﻧﺎت اﻟﺼﻮﺗﯿﺔ واﻟﻨﺼﯿﺔ ﺑﻜﻔﺎءة ﻋﺎﻟﯿﺔ. ﺑﺎﻹﺿﺎﻓﺔ إﻟﻰ ذﻟﻚ، ﺗﻢ ﺗﻘﺪﯾﻢ ﻧﻤﺎذج ﻣﺘﻌﺪدة اﻻﺳﺘﺨﺪاﻣﺎت ﻻﻛﺘﺸﺎف اﻟﻜﺎﺋﻨﺎت وﺗﺼﻨﯿﻔﮭﺎ. وﺗﻢ دﻣﺞ ﺗﻘﻨﯿﺎت ﻣﺘﻄﻮرة ﺗﺸﻤﻞ اﻟﺘﺠﻤﯿﻊ، اﻟﺘﺒﻮﯾﺐ، واﻟﺘﺼﻔﯿﺔ ﻟﺘﺤﺪﯾﺪ اﻟﻨﻤﻮذج اﻷﻛﺜﺮ ﻣﻼﺋﻤﺔ ﻻﻛﺘﺸﺎف اﻟﻜﺎﺋﻦ وﺗﺼﻨﯿﻔﮫ. ﻋﻼوة ﻋﻠﻰ ذﻟﻚ، ﺗﻢ ﺗﻘﺪﯾﻢ ﻣﻨﮭﺠﯿﺎت ﻣﺒﺘﻜﺮة ﻹﻧﺸﺎء ﻣﺤﺘﻮى وﻗﻮاﻋﺪ ﻻﺗﺨﺎذ اﻟﻘﺮار أﻛﺜﺮ دﻗﺔ. ﯾﺴﺘﺨﺪم ﺗﻘﯿﯿﻢ أداء اﻟﻨﻤﺎذج اﻟﻤﻘﺘﺮﺣﺔ ﻣﺠﻤﻮ ﻋﺎت اﻟﺒﯿﺎﻧﺎت اﻟﻤﻌﺘﺮف ﺑﮭﺎ ﻋﻠﻰ ﻧﻄﺎق واﺳﻊ وھﻲKITTI ، و nuScenes، و RADIATE، و OSU، و BPEM، و GeoTiles. ﺣﻘﻖ ﻧﻤﻮذج دﻣﺞ ﺑﯿﺎﻧﺎت اﻟﺼﻮر اﻟﻤﻘﺘﺮح دﻗﺔ ﻣﻘﺪره ﺑـ 98% ووﻗﺖ ﺗﻨﻔﯿﺬ ﻓﻲ ( 0.98 ﺛﺎﻧﯿﺔ) وھﻮ أﺳﺮع وﻗﺖ ﻣﻘﺎرﻧﺔ ﺑﻨﻤﺎذج دﻣﺞ اﻟﺼﻮر اﻟﻤﻌﺮوﻓﺔ اﻷﺧﺮى. وﺑﺪﻻً ﻣﻦ ذﻟﻚ، أظﮭﺮ ﻧﻤﻮذج اﻟﻜﺸﻒ ﻋﻦ اﻟﻜﺎﺋﻨﺎت اﻟﻤﻘﺘﺮح ﺣﺴﺎﺳﯿﺔ ﻗﺪرھﺎ 0.65 وﻣﺘﻮﺳﻂ دﻗﺔ ﻗﺪره 0.85، ﻣﻤﺎ ﯾﺆﻛﺪ ﺗﺤﺴﻦ أداﺋﮫ ﻣﻘﺎرﻧﺔ ﺑﻨﻤﺎذج اﻟﻜﺸﻒ ﻋﻦ اﻟﻜﺎﺋﻨﺎت اﻷﺧﺮى اﻟﻤﻌﺘﺮف ﺑﮭﺎ ﻋﻠﻰ ﻧﻄﺎق واﺳﻊ، وﺗﺘﻢ ﻣﻨﺎﻗﺸﺔ اﻟﻤﺰﯾﺪ ﻣﻦ اﻟﻨﺘﺎﺋﺞ ﻓﻲ ﺟﺰء اﻟﺘﺤﻠﯿﻞ اﻟﺘﺠﺮﯾﺒﻲ ﻣﻦ اﻟﻔﺼﻞ اﻟﺨﺎﻣﺲ، وھﻮ ﻧﺘﯿﺠﺔ ﺗﺤﻠﯿﻼت ﻣﻜﺜﻔﺔ أﺟﺮﯾﺖ ﻋﻠﻰ ﻋﺪة ﻣﻌﺎﯾﯿﺮ ﻟﺘﻘﯿﯿﻢ اﻟﻨﻤﺎذج اﻟﻤﻘﺘﺮﺣﺔ.

Download

Share

COinS