1.1 Billion Taxi Rides with MapD & 8 Nvidia Tesla K80s
For most of 2016 I’ve been using a dataset of 1.1 billion taxi journeys made in New York City over the course of six years to benchmark various Big Data solutions. Though these aren’t apples-for-apples comparisons, I’ve benchmarked BigQuery, Elasticsearch, Presto on EMR and Dataproc, PostgreSQL and Redshift using the same dataset.
So far all of the benchmarks have a common theme in that they are running on Intel CPUs. Probably the most dramatic difference between using GPUs versus a run-of-the-mill CPU is that memory read speeds should be able to hit 300 GB/s due to a much wider bus whereas with CPUs you’re looking at closer to 20 GB/s. This, coupled with a number of other architectural differences has resulted in some of my benchmark queries running 55x quicker than the fastest benchmarks I’ve performed in the past.
A Supercomputer Up & Running
MapD have been kind enough to grant me access to a machine that I’ll use to benchmark their GPU-based database software with the 1.1 billion taxi trips dataset. The machine I’ll be using would have been one of the world’s 20 fastest computers 10 years ago. It has 8 x Nvidia Telsa K80s, each with 2 GPUs per card. The K80 has 2.91 teraflops of double-precision performance and 8.73 teraflops single-precision performance giving me 23.28 and 69.84 teraflops of performance respectively.