Edge node processing for metadata extraction

The edge node is the interface of the SMART system to the physical world of sensor and social networks, as well as the conceptual world of the linked data cloud. As such, the edge node brings to the SMART system information:

• perceived from its physical surroundings as they are sensed by a sensor network,
• filtered from social networks,
• retrieved from the linked data cloud, and,
• inferred by combining all the above from diverse sources.

Perceived information

Perceived information comes from the environment in the form of continuous metadata streams that result from perceptual processing of the sensor feeds such as cameras, microphones, or environmental sensors. Examples of continuous metadata are the crowd density or air temperature.

Regarding the A/V signals:

• Video signal processing in SMART involves face and people tracking and visual scene analysis. For the latter, algorithms are being implemented for trafic and crowd analysis, both static (density and colour analysis) and dynamic (motion analysis and change detection).
• Audio signal processing in SMART considers audio event classification and speech processing.

The continuous metadata streams are thresholded to generate low-level events like the space is crowded or hot. The low-level events are represented as binary signals indicating the onset of the event.

Filtered information

Filtered information comes from the social networks directly in the form of low-level events. The social network manager monitors social networks like Twitter for mentions of its different aliases. It then searches for activities associated with it and subsequently monitors these activities in the social network.

The collected text is analysed to understand the association of the activity to the edge node and the details of the activity, like when or what is happening. This social information about activities is stored in the edge node database, to be expressed as low-level events when their designated start and end times come.

Retrieved linked data

Information is also retrieved from the linked data cloud. The geographical information of the node and its aliases are used to find relevant data. Unlike the social network filtered low-level events, the retrieved linked data are not time-critical. Instead they offer information that can help answering queries.

Inferred information

Inferred information comes from reasoning over the low-level events (both those from different algorithms perceiving the environment and those from filtering the social networks) and the retrieved linked data. These are the high-level events generated by running the reasoning rules.

Metadata formatting

Within SMART, we are experimenting with two formats of metadata dercription: traditional XML and RDF for linked data. Here is a working prototype of our RDF metadata description.