Covering Disruptive Technology Powering Business in The Digital Age

Databricks, the data lakehouse and Artificial Intelligence (AI) company, has announced the launch of Dolly. It is a cheap-to-build large language model (LLM) that exhibits a surprising degree of the instruction-following capabilities exhibited by ChatGPT. Using Databricks, any business can take an off-the-shelf open source LLM and give it magical ChatGPT-like instruction following ability by training it in 30 minutes on a single machine, using high-quality training data.

Dolly works by taking an existing open source 6 billion parameter model from EleutherAI and modifying it ever so slightly to elicit instruction following capabilities such as brainstorming and text generation not present in the original model, using data from Alpaca.

The model underlying Dolly only has 6 billion parameters, compared to 175 billion in GPT-3, and is two years old. This is why it is particularly surprising that it still works so well. This suggests that much of the qualitative gains in state-of-the-art models like ChatGPT may be due to focused corpora of instruction-following training data, rather than larger or better-tuned base models.

“We’re calling the model Dolly—after Dolly the sheep, the first cloned mammal—because it’s an open-source clone of an Alpaca, inspired by an LLaMA,” said Ali Ghodsi, Co-Founder of and CEO at Databricks. “We’re in the earliest days of the democratisation of AI for the enterprise, and much work remains to be done. But we believe the technology underlying Dolly represents an exciting new opportunity for companies that want to cheaply build their own instruction-following models.”

Databricks evaluated Dolly on the instruction-following capabilities described in the InstructGPT paper that ChatGPT is based on and found that it exhibits many of the same qualitative capabilities, including text generation, brainstorming and open Q&A.

Why Open Models?

There are many reasons a company would prefer to build its own model rather than sending data to a centralised LLM provider that serves a proprietary model behind an API. For many companies, the problems and datasets most likely to benefit from AI represent their most sensitive and proprietary intellectual property. Thus, handing it over to a third party may be unpalatable. Furthermore, organisations may have different tradeoffs in terms of model quality, cost and desired behaviour. Databricks believes that ML users are best served long term by directly controlling and owning their models.

The release of Dolly is the first in a series of announcements Databricks is making that focus on helping every organisation harness the power of large language models. To learn more, see HERE.

(0)(0)