LEARNER

LEARNER will introduce an automated procedure for producing training data for a Deep Learning architecture capable of detecting and associating Local Feature Points and semantic entities from completely different environmental conditions of the same scene. Modern game engines, such as Unreal 5, have become extremely good at simulating every aspect that affects a site's appearance. Thus, such an engine will be used to formulate different kinds of environments and different conditions for each one of them. All the above cases will be recorded from a simulated moving vehicle carrying an RGB-D sensor to produce a set of frame sequences for each site.

One of the innovations of LEARNER regards the development of a new AI-based SLAM approach to address mobile robotic applications in challenging environments populated by humans. In the recent litterature, most of the learning-based approaches rely on supervised schemes, which requires the formulation of vast training datasets of hand-labeled data. These methods are most often limited to environments similar to the ones they are trained on. On the contrary, within LEARNER Deep Learning methods will not substitute the model-based procedures but rather empower them by targeting the environment's representation and sensory inputs modeling. In this manner, self-supervised and reinforcement learning techniques will be used to learn a robust representation of the environment, targeting local feature points and semantic entities invariant to excessive changes to the environment's conditions and structure, without the need for labeled data. This will allow for state-of-the-art model-based SLAM methods to undertake ego-motion estimation, absolute positioning, and 3D reconstruction by taking advantage of multiple view geometry and probabilistic visual-inertial schemes.

For social-aware Path Planning, LEARNER will develop a hybrid mapping solution that will incorporate social semantics based on the humans' dynamics and their recognized action and emotional state. These characteristics will be introduced as an additional layer in the virtual force fields, inducing distinct forces that will guide the robot based on the human state. In such a manner, the middleware package's PP module will treat human subjects as a particular kind of dynamic obstacle, which will be tracked and incorporated into the SLAM mapping module.