Anand S

Anand S

Audience level:
Intermediate
Friday
noon–12:45 p.m.

Faster data processing in Python

Description

This talk will covers ways that help process and analyse visualise data faster in Python. The primary focus is on the technique (should you optimise? what to optimise? how to optimise?) while covering libraries that help with this (line_profiler, Pandas, Numba, etc.)

Abstract

Working with data in Python requires making a number of choices, ranging from the simple to the complex.

  • Should I use pickle, CSV or JSON? (Ans: CSV).
  • What do I read it with: csv.DictReader or csv.reader? (Ans: Pandas).
  • How should I parse dates? (Ans: Anything but Pandas / dateutil)
  • How do I optimise numpy calculations? (Ans: Learn vector algebra)
  • How do I run a function in parallel?
  • How to make my program restartable?
  • How do I use multiple cores?

.. and so on. This session will explain how to benchmark code and share insights on the patterns of programming that make your application faster.

Platinum Sponsors:

Silver Sponsors:

Partners: