Deep details of Python based Technology Stack at Votizen from the Inside Votizen post by Paul Stamatiou.
- Amazon Web Services
- webservers – 2 large instances running Apache, Django, Python
- load balancer – 1 large instance running HAProxy and Stunnel
- asynchronous workers – 3 x-large instances running several hundred Celery workers each
- reverse geocoding & districts – 3 x-large instances running what we call politicoder
- elastic search for voters – 2 High-Memory Quadruple Extra Large instance running elastic search with entire corpus of voters loaded into memory (verify 50-75 people/second)
- error reporting – 1 large instance running a sentry server
- utility server – 1 large instance used for various opsy operations
- db-misc server – 1 large instance running redis, memcached, and RabbitMQ
- admin server – 1 large instance used for managing deployment and monitoring site, runs CruiseControl (for deployment) and Nagios (for monitoring)
- Puppet\Chef - We were previously using Puppet to manage server configurations in our admin repository, but are in the process of moving to Chef, as our new ops person prefers Chef. Both are Ruby-based tools, but are more mature than any comparable Python solution
- Python - We write almost all our code in Python 2.6, because Python gives you wings
- Django - This is one of the most fully-baked Python web frameworks and we have 2 core contributors on our team. Version 1.4 was just released, but we are still running on 1.3
- Redis - We use redis mostly to handle caching of large lists of ids, because it has really good set/list operations built into it.
- Celery - We have to do a log of user processing after they sign up and even when they sign in: fetch contacts on your social network, find your contacts districts and see if they are voters, find your districts and see if you are a voter
- RabbitMQ - The default message queue used by Celery, and also just a good queuing system. We only use it for celery.
- Memcached - Cache whatever you can, whenever you can. Django has good memcached support and we try to take advantage of it as much as possible.
We have unit testing and continuous integration, there’s no staging environment, everything goes straight to production. We have feature flagging now to limit development features to staff.
We’re looking to move to Vagrant so the development and production environments can be nearly identical.
