Web 3.0: Installing the Plumbing

A set of behind-the-scenes technologies strives for real-time, automated intelligence gathering and analysis and could change the way we share information on the Web, writes Peter A Buxbaum for ISN Security Watch.

It may not be as viscerally exciting as Web 2.0, but there are a set of technologies working behind the scenes that are beginning to make Web research and collaboration richer and more automated. These technologies are already being exploited by US military and intelligence organizations.

The semantic web, or Web 3.0, as it is sometimes called, adds capabilities supplied by software algorithms which allow machines to understand ordinary text and, by extension, to make connections among “entities”—people, places and things—encountered when searching a body of information.
 
Web 3.0 won't dramatically change the appearance of Web 2.0 phenomena such as social networking, wikis, blogs, RSS feeds, and mashups. But it will automate some of their functions and will make searching and researching more rewarding by providing greater numbers of links to context-relevant information.

The vision for semantic searching and researching goes back to the 1960s, according to Lewis Shepherd, a former official of the US Defense Intelligence Agency and currently chief technology officer at the Microsoft Institute for Advanced Technology and Government. In that sense, it has taken quite a long time for the semantic web to get up to speed, but that is exactly what is happening now.

“The semantic web will be a complex, boring, but very empowering set of capabilities,” Shepherd told ISN Security Watch. “It will provide people with a rich networked and hyperlinked dense array of semantically enabled text. It will change the way people collaborate and encounter information on the web.”

The US intelligence community already uses social software to form collaborative project groups. “Products like the President's Daily Brief are different than just five years ago because of the multi-agency input powered by social software,” said Shepherd.

With semantic capabilities, these groups will become self-forming and self-evolving. “The composition of the team will be drawn from the data itself,” said Shepherd. “You won't have to draw on databases to search for people using keywords. You could have communities dynamically self-forming for research projects with its composition changing automatically over time. It will still be controllable by users but they will have these tools as additional options.”

Similarly mashups - integrated presentations derived from multiple data sources - will automatically be linked to relevant material to provide richer context to the information being presented. Mashups are created by using middleware to extract data from databases and display them on internet browsers in a variety of formats.

In the military and intelligence environments, human intelligence feeds are combined with mapping and global positioning systems to create information visualization. The intelligence community uses mashups to display data provided by the US National Geospatial-Intelligence Agency from Google Earth and Arc GIS Explorer, according to Shepherd.

“The value of Web 2.0 mashups will be improved in Web 3.0 if semantic meanings are baked right into the data,” said Shepherd.

Real time, really quick

The US Defense Intelligence Agency has deployed a several different commercial metadata extraction and tagging services in order to inject greater semantic meaning into its data. This extraction and tagging “factory” works 24/7 on the Agency’s data traffic to produce semantically enabled markup of data and text.

The US Air Force Air and Space Intelligence Center, located at Wright-Patterson Air Force Base in Ohio, “makes heavy use of XML data and semantic cataloging of entities which they have built directly and organically into their analytic and production processes,” said Shepherd. The center uses that capability to “pull together disparate streams of data and visualize the information on the fly in a variety of impressive ways” which are customized to the needs of users such as analysts and policymakers.

Assembling custom-made and highly useful intelligence packages in real time - on the fly - is a recurring theme in the military and intelligence use of semantic capabilities. The US Defense Advanced Research Projects Agency recently awarded a contract to software developer Modus Operandi to develop a system that will enable the military to quickly boil down large amounts of information and use it to support tactical operations.

The first phase of the project is designed to use advanced semantic techniques to automate the study of large amounts of textual data and discover patterns and clues that will provide warfighters with near real-time situational awareness, according to Richard Hull, the company's chief scientist.

Hull envisions the system to be used for missions such as hostage rescues, insurgent extractions and targeting, where a military unit must assess a tactical situation and act quickly.

"We now have the capability, through scalable semantic algorithms, to very quickly study large amounts of information,” he said. “It will provide far more actionable information to warfighters and give them more confidence in their decision-making during very difficult tactical operations.”

The Modus Operandi system will extract intelligence from the unstructured data of natural language texts such as news stories, blogs and press releases. Applying semantic algorithms, the system will automatically study large volumes of text almost instantaneously to offer insight into a particular tactical situation.

From fantasy to fact?

Another aspect of delivering real-time intelligence involves the Pentagon’s dream of a “push” information paradigm, in which analysts, decision makers and warfighters will automatically be provided the information and intelligence they need without having to search for it.

“We now still rely on the pull paradigm in which users are composing and performing complex queries in their hunting and gathering efforts,” Shepherd said. “What users prefer are systems smart enough to semantically understand raw text, to correlate multiple types of information, and to compose relationships across all sources of information.”

The semantic web will enable a system which would provide analysts, commanders and warfighters with prepackaged actionable intelligence packets. “Once the system understands peoples, places, things and events mentioned in texts and the relationships between them, it will be able to route real time information to where it is needed,” Shepherd said.

All of which is still a work in progress and has been since the 1960s fantasy of a semanticized universe. What the architects of the semantic web eventually realized, said Shepherd, is that it couldn't be formed with one big bang. Instead it required the lengthy building out of a “tremendous amount of plumbing.”
JavaScript has been disabled in your browser