Google's first TPU was designed to run neural networks quickly and efficiently but not necessarily to train them, which can be a large-scale problem. So the second TPU, which Google describes as a Cloud TPU, which can both train and run machine learning models.
Google doesn't appear to have said very much about the devices except that each Cloud TPU can provide 180 TFLOPS of floating-point performance and that the TPU2/Cloud TPU has been designed to work well together in large arrays.
It is not clear whether a TPU is a single ASIC or four identical ASICs on a PCB as shown in the picture at the top of this article. What Google has said is that 64 Cloud TPUs can be put together to form something called a TPU pod capable of up to 11.5 petaflops.
Skyscraper heatsinks to cool how much power consumption? Source: Google.
As a benchmark of performance Google said that a large-scale translation model can run in an afternoon on 8 TPU-2s compared with a full day on 32 of the best commercially-available GPUs. So that represent about a factor of ten improvement in performance.
It is also interesting to note that the TPU-1 was benchmarked at a maximum performance of 92TOPS, which happen to be 8bit integer operations in a systolic array, while the Cloud TPU is benchmarked expressly in terms of floating-point operations.
But what sort of floating-point operations? Full 32bit precision; 16bit half-precision? Is quantization down to 8bit integer still an essential part of the architecture?
As would be expected these second generation TPUs can be programmed with TensorFlow, an open-source machine learning framework available from the GitHub repository.
Google also announced that it would make 1,000 of its Cloud TPUs available to machine learning reseachers for free via something called the TensorFlow Research Cloud.
While we don't yet know much about the Cloud TPU Google has provided more detail about its first TPU and this may indicate areas where the Cloud TPU is likely to be as good or better.
Next: What about TPU-1?