DEDALE: Mathematical tools to help navigate the Big Data maze

Big Data have become essential for science and technological advancement. The DEDALE project is developing new analysis methods to work on them.

Managing the huge volumes and varying streams of big data digital information presents formidable analytical challenges to anyone wanting to make sense of it. Consider the mapping of space, where scientists collect, process and transmit giga-scale data sets to generate accurate visual representations of millions of galaxies. Or consider the vast information being generated by genomics and bioinformatics as genomes are mapped and new drugs discovered. And soon the Internet of Things will bring millions of interconnected information-sensing and transmitting devices.

Transforming all this data into usable information and knowledge requires exceptional processing capacity that is not easily available. So how can big data challenges be addressed?

One way is to make the presentation of data as efficient as possible. This is the approach followed by DEDALE – a project funded under the EU’s Future and Emerging Technologies (FET) programme – which stands for ‘Data Learning on Manifolds and Future Challenges’ and also references Daedalus (‘Dédale’ in French), the mythological Greek craftsman and inventor who famously solved the riddle of running a length of string throughout a spiral shell by using an ant.

The same problem-solving spirit of the Greek innovator has been used in the DEDALE project to develop new digital mathematical tools that can automatically identify hidden patterns in large-scale data sets, aiding the mapping of data-intensive, geometrical space such as the countless galaxies present in space.

The challenges that astrophysicists face to visually represent space typically include the unavoidable distortions (’background noise’) that arise during observations, such as the optical imperfections due to the movement of a telescope, or tiny changes in atmospheric conditions. However, with the algorithms and compression techniques that DEDALE is developing, not only can these distortions be removed to leave pristine galactic images, they could also help identify where space material that scientists cannot directly observe due to the absence of light — dark matter – may be located.

The use of DEDALE’s tools may not only be limited to the study of outer space. They could prove just as useful to explore the brain. The technology could help neuroscientists to deal with similar data-related challenges as those faced by astrophysicists. In fact, neuron representations are computationally highly demanding (they consume a lot of power, memory and storage) and very expensive. DEDALE’s tools can therefore help to reduce research costs by making data processing more time efficient.

The tools could also be adopted for real-time terrain mapping. Data sets from drones, satellites and other devices could be efficiently complemented using DEDALE’s technology.

These scenarios are only the beginning. Like the ant guiding the string through the spiral shell, the DEDALE project will hopefully guide scientists through the ever-expanding big data mazes of the 21st century.

Source: https://cordis.europa.eu/news/rcn/128979_en.html?isPermaLink=true&WT.mc_...