A Multimodal Event processing engine to Analyse Complex Event Patterns in Real Time

GNOSIS is a Multimodal Containerized framework that blends neural (DNNs) and symbolic (rules) models within the event-based paradigm to process IoMT multimodal streams utilising hybrid processing through video event knowledge graphs. GNOSIS combines deep learning and computer vision with real-time event processing capabilities within a microservice-based architecture for video stream processing and AI analytics in Complex Event Processing scenarios. GNOSIS simplifies the querying of streams and the deployment and optimisation of multiple pipelines.

Multimodal Event Processing Engine

MEP provides a formal basis for native approaches to multimodal content analysis (e.g., computer vision, linguistics, and audio) within the event processing paradigm to support real-time queries over multimodal data streams. MEP enables the user to process multimodal streams under the event-processing paradigm and simplifies the querying of multimodal event streams. 

Learn More

Techniques

VEKG

Video Event Knowledge Graph to Represent Video Streams for Complex Event Pattern Matching

Learn More

VIDCEP

Complex Event Processing Framework to Detect Spatiotemporal Patterns through user defined queries.

Learn More

Multimedia Pub/Sub

High performance Multimedia Event Processing based on the deep convolutional neural network (DNN) techniques

Learn More

Online Training

Supporting queries of seen as well as unseen subscriptions of multimedia events

Learn More

Microservices Architecture

Appropriate use of microservices for independent deployablility

Learn More

Applications

Occupational Health and Safety 

 learn more

Smart Traffic Management

 learn more

Smart Agriculture

 learn more

Activity Recognition

 learn more

Pose Detection

 learn more

Features

Power of Graphs

Powered by Python’s graph library, GNOSIS creates complex network of detected objects to find complex patterns

Handle Multiple Video Streams

GNOSIS can process multiple video streams in near real-time with expressive reasoning over streams.

Containerisation

GNOSIS provides Docker container for quick and efficient deployment of the framework.

Suite of Video Query Operations

GNOSIS allows users to express high-level queries to process complex events in video streams.

Monitoring

GNOSIS uses Jaeger to trace and troubleshoot problems during processing.

Multimodal Processing

GNOSIS processes Multimodal Event Knowledge Graphs to provide a knowledge representation for multimodal event streams’ semantic, spatial, and temporal content.

News

Demo for VLDB Endowment 2021

Yadav, Piyush, Dhaval Salwala, Felipe Arruda Pontes, Praneet Dhingra and Edward Curry (2021). “Query-Driven Video Event Processing for the Internet of Multimedia Things” Proceedings of the VLDB Endowment (VLDB) Demo[…]

Read more
September 13, 2021 0

Research Study accepted in VLDB Endowment 2021

Yadav, Piyush, Dhaval Salwala, Felipe Arruda Pontes, Praneet Dhingra and Edward Curry (2021). “Query-DrivenVideo Event Processing for the Internet of Multimedia Things” Proceedings of the VLDB Endowment (VLDB) 14(12)

Read more
July 29, 2021 0

About Us

The Open Distribued Systems (ODS) unit at the Insight Centre for Data Analytics, NUI Galway investigates novel techniques to contribute to decentralised and open distributed infrastructure. In collaboration with scientific and industrial partners the domain will identify and develop techniques to impacts people and organisation.

Areas of work: Event processing, Middleware, Smart cities, Incremental data management, Publish-subscribe

Dr Edward Curry

Research Leader

Dr Amin Anjomshoaa

Research Fellow

Muhammad Jaleed Khan

PhD Researcher

Felipe Arruda Pontes

MSc Researcher

Dhaval Salwala

Research Assistant

Dr Piyush Yadav

Postdoctoral Researcher

Tarek Zaarour

PhD Researcher

Atiya Usmani

PhD Researcher

Niki Pavlopoulou

PhD Researcher

Praneet Dhingra

Research Assistant