SO Jetpack Stats Only is a fork plugin of Jetpack and it only enables the Stats Module enabled. You can use this plugin to know your Jetpack Stats without installing Official Jetpack plugin with so many modules.
Simply install and activate SO Jetpack Stats Only plugin. Upon activation visit your admin area Dashboard -> Settings -> SO Jetpack Stats Only admin page, connect it with your WordPress.com account and then you can view your site stats just like you do with official Jetpack plugin.