Andrew Godwin

Andrew Godwin

Audience level:
Intermediate
Wednesday
9 a.m.–1 p.m.

Datastores, Python and You

Description

Basic data storage is easy enough - relational databases, lines in flat files. What about the other options, though? There's a whole range of storage options and formats out there, and we'll go through lots of worked examples, exploring both how easy they are to write and manage as well as how they fit the different kinds of data, and try out a few stranger options along the way.

Abstract

Storing data is one of the key parts of programming - and sadly, often one of the least diverse in terms of the solutions programmers often use. Relational databases are good heavy lifters, but there are plenty of other options - and a whole load of formats for storing and modelling data that could make your next project significantly easier.

The tutorial will be an exploration of several different storage options, with hands-on segments where you can try them yourself and explore the strengths and weaknesses of each option. Among other things, we'll look at relational databases (PostgreSQL), key/value stores (Redis and Riak), queues (Kafka), graph databases (Neo4j), flat-file formats (SQLite, Protocol Buffers), search engines (Elasticsearch) and more - including a look at how filesystems themselves are implemented and some historic databases to see how we've evolved.

Participation is optional - there'll be several worked examples you can follow along with if you have the software (there may also be a server in the room you can connect to), but you can also follow along with the presenter if you don't wish to type or don't have easy access to a Mac or Linux computer (some examples may not work on Windows).

The session aims to give you both a better idea of the universe of storage formats and options as well as help you get a better understanding of how storage actually works, to help you make more informed decisions and better-designed architectures in the future.

Platinum Sponsors:


Silver Sponsors:


Partners: