Python is actually a great language for building Activity Streams and Newsfeeds. Tommaso and I have written the Stream Framework package.
https://github.com/tschellenbach/stream-framework
It is currently the most used Python solution for building newsfeeds. We re also offering a hosted solution at https://getstream.io. The Django client is by far the easiest to get started with:
https://github.com/GetStream/stream-django and python can be found here (https://github.com/getstream/stream-python)
The templating part works like this
{% load stream_django %}
{% for activity in activities %}
{% render_activity activity %}
{% endfor %}
This will render a template located in activity/tweet.html with the activity as context. For instance
{{ activity.actor.username }} said "{{ activity.object.body }} {{ activity.created_at|timesince }} ago"
The full docs are here:
https://github.com/GetStream/stream-django#templating
The Stream Framework allows you to build any type of newsfeed using either Redis or Cassandra. It s build to scale and creates the individual newsfeeds using a fanout process.
Besides the Stream Framework (which I obviously prefer) there are many other solutions out there. A full list is available on django packages:
https://www.djangopackages.com/grids/g/activities/
Note that with newsfeeds there are a few scaling issues to keep in mind.
In general there are 3 common approaches:
Denormalization strategies
Pull
Most users start out this way. When you open the feed page you simply query the feeds from all the users you follow. If the users feeds are stored in memory this will keep on working for quite some time. Eventually it is quite hard to keep on using such as strategy though as you often have to query most of the nodes storing your user s feeds.
Push
The push approach writes your activity to all of your followers feeds. Of course this means you re wasting a ton of resources, but the end result is a pre computed feed per user. This approach (though initially not very efficient) scales nicely.
Combination
Some optimized systems use a combination of these two approaches. Also see the Yahoo paper on this topic.
Storage options
In terms of storing all this data the most common options are Redis, Cassandra and MongoDB.
Let s quickly compare these:
Redis
Redis is Extremely easy to setup and maintain. It however only stores data in memory. This means you ll have to optimize how you serialize data and maybe fallback to the database for less frequently queried data. Another problem is that it s not trivial to add machines to your Redis cluster.
MongoDB
Mongo DB is used primarily by a few ruby projects and it s also available as a backend for pump.io by e14n. I ve personally never ran it in production so I can t properly evaluate this option. There are however a lot of blogposts covering issues with the performance, scalability and maintainability of mongo.
Cassandra
Fashiolista, Instagram and Spotify are all using Cassandra. Our hosted solution also uses Cassandra as a backend. It s extremely cost effective to operate and you can add more nodes with ease. The only problem is that it s hard to setup and maintain.
Articles
In addition have a look at this high scalability post were we explain some of the design decisions involved:
http://highscalability.com/blog/2013/10/28/design-decisions-for-scaling-your-high-traffic-feeds.html
To learn more about feed design I highly recommend reading some of the articles which we based Feedly on: