PhD Thesis : develop advanced models and techniques for the optimal fusion of GNSS, video, IMU, odometer and magnetometer data

This PhD thesis is co-funded by CNES and THALES ALENIA SPACE (pending final acceptance by the CNES PhD selection committee). This PhD thesis can only be applied through the following online application:

https://recrutement.cnes.fr/en/annonce/2701729-24-319-performance-prediction-of-gnssimuvideo-position-nominal-error-31500-toulouse

GNSS (Global Navigation Satellite Systems) systems are widely used for navigation. However, in certain environments such as urban areas or inside buildings, the processing of received GNSS signals to obtain a position estimate can be degraded or impossible. To overcome these limitations, it is interesting to combine the observations from the GNSS receiver with other sensors, in a hybrid navigation approach.

The combination of GNSS with IMU, odometer or magnetometer information is a first step providing interesting performance, but in the situation of using a low-cost MEMS IMU, the hybrid performance can still have limitations if GNSS reception is degraded for a few tens of seconds because of IMU or odometer bias/drift or because of magnetic interference to the magnetometer. The hybridization of GNSS with IMU, odometer and magnetometer, and the geometric processing of images from a video camera is a promising approach that can possibly maintain accuracy at acceptable levels even when IMU drags the position error with an increasing drift. Video processing allows extracting user trajectory parameters even if no a priori database is used and may help reducing the drift of the poorly recalibrated IMU position if GNSS is degraded. In this case, each sensor has complementary advantages and disadvantages. The challenging cases, including high-rise buildings, low illumination, fast movement and highly dynamic environments, make it impracticable for a single-sensor based algorithm to provide continuous reliable position estimate. To our knowledge, this GNSS/Video/IMU combination has already been exploited by various authors (ex: [1], [2], [3]) with good results, even though in situations where GNSS RTK-PPP or high-grade IMU is impracticable, the resolution of the accumulated drift is sometimes difficult, but complete characterization of the error is not provided as a general result but from collected datasets, and the main unknown at this stage if the performance of the video-extracted geometric observations.

The featured complementary sensor in this combination is the video sensor, which is expected to bring user trajectory information from the image sequence, therefore in a manner which is not dependent on reception of RF signals, and which is not a pure dead reckoning building up errors. However, scene environment, scene luminance, camera technology and tuning, user dynamics, geometric processing of images will strongly influence the quality of the trajectory information brought by the camera to the system. The geometric processing of the video can be accomplished via line-following, visual odometry, perspective-and-point, and SLAM. The favoured technique at this stage is SLAM, which allows extracting visual landmarks in the observed scene for the joint estimation of the position of these reference points and the position and orientation of the camera. A full state of the art review will complete the landscape of applicable techniques. SLAM has many setting parameters for initialization, feature detection, feature description, feature matching, feature tracking, key frame identification, local and global optimization, and its camera pose estimation error performance is known to vary upon these parameters in size and in correlation time. In addition, camera tuning such as frame rate, field of view, shutter, resolution, focus, gain, can influence the resulting errors.

To our knowledge, integrity bounding of the position error of such a combination, i.e. reliably predicted 95%-accuracy, integrity, continuity and availability performance of such a combination is not available in the public literature in most of the cases, except is some well bounded cases [4]. Several long steps are needed to reach that integrity goal, including development of performance requirements for applications, identification of nominal measurement error models including correlation time, identification of failure modes, development of monitors with associated detection thresholds, prediction of monitors missed detection performance and minimum detectable biases, determination of accuracy and integrity bounds on position error, prediction of availability and continuity performance. This proposed thesis is an additional step towards the goal of integrity bounding, but is not reaching that overall goal of integrity.

To derive the measurement error models, existing or new data collections will be used to infer the shape, the parameters of the models and in particular the description and the limitations of applicability for the environment, but it is felt that the performance prediction cannot rely only on data collection which has inherent limitation in extension of cases, plus inherent unknowns about the truth of the values of several parameters like the scene and the conditions of visibility.

The purpose of this thesis is to develop advanced models and techniques for the optimal fusion of GNSS, video, IMU, odometer and magnetometer data in order to obtain an accurate and robust hybrid positioning system in constrained environments and to reliably predict a 95% position error bound in a representative set of conditions. As a by-product, it is also proposed to analyze further the velocity error models and distributions.

The expected original contributions are:

1)     The development of nominal error models and models of video measurement geometric anomalies resulting from video processing as a function of descriptive variables (environment, visibility, travelled distance, …). These models will be inferred from existing or collected real data, and will also be checked against real data.

2)     The development of a data fusion algorithm tuned on the basis of these models. The starting point is the existing combination algorithms

3)     The development of a GNSS/Video/IMU positioning simulator reliably implementing the developed models and the combination algorithm.

4)     The analysis and the prediction of a reliable 95% accuracy performance bound based on the descriptive variables for a variety of defined situations, and the comparison with other state-of-the-art approaches.

5)     As a by-product, the analysis of the velocity error models and distributions.

The scientific challenges include in particular the temporal synchronization of the different sensors, the proper identification and the management of their nominal measurement uncertainties and the definition of their descriptive variables, 95% robust position estimation, and 95% position performance prediction.

The targeted applications concern autonomous navigation in urban environments, with some tailoring to some types of applications and environments like airport surface movement by vehicles. The expected benefits are both scientific, through methodological contributions, and technological for the industrial partner.

[1] Ben afia, Amani. Development of GNSS/INS/SLAM Algorithms for Navigation in Constrained Environments. PhD, Signal, Image, Acoustique et Optimisation, Institut National Polytechnique de Toulouse, 2017

[2]  Shaozu Cao, Xiuyuan Lu, and Shaojie Shen, GVINS: Tightly Coupled GNSS-Visual-Inertial Fusion for Smooth and Consistent State Estimation, IEEE Transactions on Robotics ( Volume: 38, Issue: 4, August 2022)

[3] Angelino, Cesario Vincenzo, Baraniello, V.R., Cicala, Luca, UAV position and attitude estimation using IMU, GNSS and camera, 5th International Conference on Information Fusion, FUSION 2012

[4] J. Vezinet, Study of Future On-board GNSS/INS Hybridization Architecture, INPT Ph.D. Thesis, 2014

Master’s degree in electrical/telecommunications/electronic engineering with background in estimation theory and signal processing. Knowledge in GNSS and aerospace systems will be appreciated.

ENAC facilitates the integration of workers with disabilities