Microsoft, Google and Baidu may be competitors in the business world, but when it comes to open-access academic resources, they’re all working together – thanks to a collaboration created by Seattle’s Allen Institute for Artificial Intelligence.
The Open Academic Search working group, or OAS, was set up to unite a wide spectrum of researchers working on academic search tools.
“It’s a number of connected initiatives, but all centered on how we promote discovery,” said Marie Hagman, OAS product manager as well as product lead for Semantic Scholar at the Allen Institute.
The institute, better known as AI2, was launched by Microsoft co-founder Paul Allen in 2014 to conduct research in AI and computer science for the common good.
The internet was created as a tool to link up academic resources, but the task of finding the right resources is becoming increasingly challenging because the volume of information is rising so quickly.
Three years ago, researchers reported that there were well more than 100 million scholarly English-language documents on the Web, and that estimate is thought to have doubled just since then. Indexing those documents, in all their various formats, poses a grueling test for AI agents and search engines.
Hagman said the creators of academic search tools – including Semantic Scholar as well as Google Scholar, Microsoft Academic and China’s Baidu Scholar – all have a basic set of requirements they have to work through.
“That work is not where people are differentiating their products,” she said. “It’s just the hard work we all have to do in order to have anything at all.”
This is where the Open Academic Search initiative can facilitate the sharing of metadata, user behavior data and other resources.
“Once you’ve done the basics, there’s no shortage of opportunities for people to innovate,” Hagman said.
In addition to Microsoft, Google and Baidu, OAS’ advisory board includes representatives from academic institutions such as the University of Washington and the University of Washington; databases such as ArXiv, ACM Digital Libraries and Wikimedia; and research funders such as the Bill and Melinda Gates Foundation.
The Gates Foundation requires the researchers they support to make their studies freely available – which Hagman says has given a powerful boost to the open-access movement, even at publications that traditionally put their content behind a paywall. As more sources add to a rising tide of open-access research, database tools will have to keep on top of the wave.
To advance the cause, AI2’s Semantic Scholar team recently released several tools to accelerate innovation in academic search: Open Research Corpus, ScienceParse, PDFFigures and the Semantic Scholar API.
