Another deep learning processor appears in the ring: Grayskull from Tenstorrent – Electronics Weekly

It describes the technology behind the processor as: The first conditional execution architecture for artificial intelligence facilitating scalable deep learning. Tenstorrent has taken an approach that dynamically eliminates unnecessary computation, thus breaking the direct link between model size growth and compute/memory bandwidth requirements.

Conditional computation?

Conditional computation enables adaptation to both inference and training of a model to the exact input that was presented, like adjusting NLP model computations to the exact length of the text presented, and dynamically pruning portions of the model based on input characteristics, is how the company describes it.

Grayskull integrates 120 Tensix proprietary cores with 120Mbyte of local SRAM. It has eight channels of LPDDR4 for supporting up to 16Gbyte of external DRAM and 16 lanes of PCI-E Gen 4.

The Tensix cores have a packet processor, a programmable SIMD and maths computation block, five single-issue RISC cores and 1Mbyte of ram.

Associated software model

The array of Tensix cores is stitched together with a double 2D torus network-on-chip, which facilitates multi-cast flexibility, along with minimal software burden for scheduling coarse-grain data transfers, according to the company.At the chip thermal design power required for a 75W bus-powered PCIE card, Grayskull achieves 368Tops and up to 23,345 sentence/second using BERT-Base for the SQuAD 1.1 data set.

According to the Tenstorrent:

For artificial intelligence to reach the next level, machines need to go beyond pattern recognition and into cause-and-effect learning. Such machine learning models require computing infrastructure that allows them to continue growing by orders of magnitude for years to come. Machine learning computers can achieve this goal in two ways: by weakening the dependence between model size and raw compute power, through features like conditional execution and dynamic sparsity handling, and by facilitating compute scalability at hitherto unrivalled levels. Rapid changes in Machine learning models further require flexibility and programmability.

Claimed Grayskull benchmarks

Grayskull is aimed at inferencing in data centres, public cloud servers, private cloud servers, on-premises servers, edge servers and automotive.

Samples are said to be with partners, with the processor ready for production this autumn.

The Tenstorrent website is here

Read the rest here:
Another deep learning processor appears in the ring: Grayskull from Tenstorrent - Electronics Weekly