Anthropic, an AI research company established by OpenAI dropouts in 2021, is reportedly planning a model that is ten times more powerful than the current most powerful AI systems. The company expects to generate up to $5 billion in revenue over the next two years through the development of a frontier model that is estimated to cost around $1 billion in the next 18 months. The model, provisionally named Claude-Next, will be in the order of 10 to the power of 25 floating point operations (FLOPs), several orders of magnitude more than other systems known today. The company is committed to AI security, focusing its work on alignment research, automated red teaming, constitutional AI, bias removal, reinforcement learning with human feedback, and robust, helpful, harmless AI systems.
The development of large AI models has caused concerns about AI safety, with Anthropic stating that current technical advances could trigger a major AI revolution that could cause a disruption that tempts corporations and countries to operate insecure AI systems under time pressure to keep up. Anthropic considers two imbalances to be conceivable in such a situation: either AI systems could “strategically pursue dangerous goals”, or the systems could make harmless errors – but in situations in which they have a serious impact. According to the company, both could “lead to a catastrophe.”
Despite Anthropic’s commitment to AI security, the company’s investors include several tech giants that have signed an open letter demanding a break in the development of large AI models. Although this contradicts Anthropic’s plans to construct significantly more powerful models in record time, the company aims to align AI systems with human intentions so that they respond to questions and tasks following a set of guidelines and guiding principles.
Anthropic has raised almost $1 billion in venture capital since its inception and is said to be looking to raise another $300 million in a recent round of funding. The company’s total capital will be $1.3 billion at an estimated company valuation of $4 billion.
