While there has long been a concern—fear, even—that artificial intelligence (AI) could pose a threat to humans, especially in the jobs department, there is still a strongly held belief that the technology will complement and enhance human capabilities, not replace them.
AI is already being tapped to help with any number of things, such as flagging abnormalities in medical images, delivering telemedicine solutions, detecting fraud and analyzing behavior in financial transactions, translating between languages, and automating processes.
When it comes to the art of writing, the process of putting pen to paper (or fingers to keyboard to screen) has not changed much. A new application developed by a young Israeli startup founded by some of Israel’s leading minds in AI is here to help us elevate our abilities in this field.
“The way we write hasn’t really changed since the 1980s,” says Yoav Shoham, a computer scientist, serial entrepreneur, and Stanford professor emeritus, whose company AI21 Labs developed an innovative new Chrome extension called Wordtune. “We believe the next phase is to make it an AI-first experience, a collaboration between the person and the machine,” he tells NoCamels via email.
AI21 Labs was co-founded in 2018 by Shoham; Ori Goshen, a former member of the famed 8200 tech unit of the IDF and a serial entrepreneur; and Amnon Shashua, professor at the Hebrew University, and co-founder and CEO of Mobileye and OrCam Technologies.
The firm emerged from stealth mode late last year to announce a significant funding round led by Pitango First, the seed and early-stage fund of Pitango’s investment platform, and to launch its first product—Wordtune.
According to the company, Wordtune is the first AI-based writing companion that understands context and meaning. It does this by leveraging advanced AI tools and language models to understand the context and semantics of written text.
“Wordtune ushers in a new writing paradigm, where humans and machines work iteratively together to distill thoughts and shape them into words,” Shoham said in the announcement of the launch.
The free Google Chrome browser extension provides suggestions to users looking for the right words. It offers sentences in a variety of lengths, tones, and styles to help users find the best word for their context. Premium offerings are also available.
“There are many tools that can correct your grammar and spelling but there’s nothing in the market with deep AI—something that captures your thoughts, and truly understands context and meaning,” Shoham tells NoCamels. “That’s where Wordtune comes in. Wordtune offers users ways to translate their thoughts into quality text, so their writing is authentic, clear, and compelling.”
Wordtune can “reduce the gap between your thoughts and what you end up writing,” Shoham explained. “We set out to build this deep technology, and use it to reinvent how we write and read. We worked on it for over two years,” he says.
More broadly, AI21 Labs is working on a number of offerings that will enable machines “to become true thought partners.”
Shoham, an entrepreneur who previously sold two companies to Google, is also the initiator of the AI Index, a project launched publicly at the end of 2017 to track activity and progress in AI.
He tells NoCamels that he met Goshen, his co-founder and co-CEO, through his wife, Noga Goshen, who is the co-founder and managing director of We Code, an academic coding bootcamp for young adults from disadvantaged backgrounds. Shoham led the bootcamp alongside Noga Goshen and Shimon Schoken, a professor of computer science at the IDC Herzliya and co-founder of Israeli edtech startup Matific.
According to Shoham, Shashua joined AI21 Labs shortly thereafter. AI21 Labs is also made up of leading AI experts and professors from around the world as well as veterans of the elite IDF 8200 tech unit, who serve the company in various capacities.
Artificial Intelligence for writing
While Wordtune is AI21 Labs’ first official product, the company has also released demos of other models it is working on. In August 2019, AI21 Labs released a paper called SenseBERT: Driving Some Sense into BERT, in an effort to teach algorithms to better understand the ambiguities of human language.
The research was based on Bidirectional Encoder Representations from Transformers (BERT), a Transformer-based machine learning technique for natural language processing (NLP) pre-training developed by Google. BERT was created and published in 2018 by Jacob Devlin and other colleagues from Google. Google has been leveraging BERT to better understand user searches since 2019.
“In this paper, we bring forth a novel methodology for applying self-supervision directly on the level of a word’s meaning,” the authors wrote. “By infusing explicit word-sense information into BERT’s self-supervision signal, we expose the model to lexical semantics when pre-training on a large unannotated corpus. We name the resultant sense-informed model SenseBERT.”
Alongside SenseBERT, AI21 Labs also announced HAIM, an acronym that came from “Halfway Acceptable Interpolating Machine.” HAIM is a language model that can fill-in synthetic text between a human-written beginning and a human-written ending. The demo was released as a step towards controllable text generation. The company has since released HAIM-1.5, which is the next generation of HAIM, or what the company calls “HAIM on steroids.”
HAIMKE, which the company calls an evolution on its previous work with HAIM and HAIM-1.5, is a language model that generates documents from human-written bullet points, converting them into written paragraphs.
These models demonstrate how a truly collaborative human-machine writing interaction could work.
The company’s approach is to let the model generate an initial text and let the user take it from there with manual editing or regenerating parts.
As AI21 Labs continues to bridge the traditional machine learning technique of knowledge representation methods (extracting semantic predictions from language) with deep neural networks, it remains both a deep R&D lab and a product company. The company employs about 40 people at its Tel Aviv offices.
“We’re going to continue building cutting-edge AI technology and add additional category-defining products,” Shoham says.
This article first appeared in NoCamels, which covers innovations from Israel for a global audience.