Etched Startup announced a new high-speed, transformer-specialized chip – Sohu

An American startup called Etched announced a new Chip that they are going to release under the name Sohu. The chip is strictly specialized for transformer AI-models, which are the most widely used in today’s market, and is manufactured using TSMC's 4 nm process technology.. The transformer architecture is burnt directly into the chip, dramatically improving its efficiency for this particular model while making the chip unsuitable for other models like CNN, LSTM, or SSM. According to Etched, a 8x Sohu server can be as powerful as a 160 NVIDIA H100 server.

Sohu processor is announced to become 10 faster and cheaper than NVIDIA Blackwell chips. For example, a Sohu server can process Llama 70B tokens 20 faster than (H100 server – 23 000 tokens per second), and 10 faster than a B200 server (~45 000 tokens per second).

The point behind creating such a chip is the slowing down of Moore's law, leading to general-purpose chips' lower efficiency. What major chip manufacturers were currently doing was increasing the chip size. The only solution that Etched sees is to focus on specialized chips that can be much cheaper, easier to produce and more efficient. The similar approach took place in the case of ASIC chips for crypto mining, which proved to be 10-100 times faster than regular GPUs.

The main principle behind the productivity of Sohu chips is the simplification of hardware and software pipeline – the manufacturers remove anything irrelevant for the main goal both when it comes to software and hardware components, making the chip highly specialized.