“The Street Preacher” - A Hyper-Local Twitter Bot

I walk through Yonge & Dundas Square in Toronto every day.

So. Many. People.

That intersection, which some call Toronto’s equivalent of Times Square, has a large number of street preachers. Loud, startling, obnoxious people that yell warnings of doom or urge repentance. Silly people.

I decided to use Twitter’s real-time streaming API to make an extremely specific location-based Twitter bot. The purpose? To respond to you if you tweet near the street preachers at Yonge & Dundas, with similar messages. Call it art, or a statement about society, or making fun of those preachers, whatever - I call it a fun technical and social experiment.

Using an excellent ArsTechnica article as a guide, I created a quick Python script that watches the Twitter stream for a given area, and replies to tweets in a very specific location. (±10 meters or so, by my guess.) If you’re one of the lucky few to tweet within those bounds, you’ll get a reply from @yonge_dundas:


A day later, I decided to clean up the script (rewrite it in Ruby, too) and open-source it. Well, here it is, in a quick Github gist:

# "The Street Preacher"
# hyper-local twitter bot
# by Peter Sobot (psobot.com)
# November 29, 2011
# ---------------------------
# Instructions:
#   Place phrases in phrases.txt (one per line)
#   Set a latitude and longitude (@from_lat, @from_lng)
#   Set a radius (currently in degrees lat/long)
#   Set your twitter account's username (to prevent feedback)
#   Enter your Twitter application keys and OAuth credentials
#     (get them from https://dev.twitter.com/)
#   Run:
#     `ruby preacher.rb start`
#   ???
#   Profit! (Not really.)
# ---------------------------
# Defaults are set to Yonge & Dundas Square, Toronto.
# @yonge_dundas is a twitter clone of the notorious street preachers
# that live at that intersection.
# ---------------------------

require 'rubygems'
require 'tweetstream'
require 'twitter'
require 'logger'

PHRASES = Dir.pwd + '/phrases.txt'
EXCLUDED_USERS = Dir.pwd + '/excluded_users.txt'
USER_TWEET_TIME_LIMIT = 10800 #in seconds, how often is too often? 3 hours.
ERROR_TIMEOUT = 300           #in seconds, how long do we wait if Twitter barks at us?

# Set this to your account's username, so it doesn't feedback loop.
USERNAME = 'yonge_dundas'

@logger = Logger.new STDERR

# Twitter phrases:
@phrases = IO.readlines(PHRASES).collect{|p|p.chomp}.compact.reject{|n|n.empty?}
def random_phrase
  @phrases.sort_by{ rand }.first

def excluded_users
  IO.readlines(EXCLUDED_USERS).collect{|p|p.chomp}.compact.reject{|n|n.empty?} + [USERNAME]

# Let's store a list of people and times we've tweeted at them, to avoid spam
@user_cache = {}

def hit_recently user_id
  !@user_cache[user_id].nil? && (Time.now - @user_cache[user_id]) < USER_TWEET_TIME_LIMIT

# Tweet from:
@from_lat = 43.65641564830964
@from_lng = -79.38105940818787
radius = 0.001 # catch area in degrees lat/lng

consumer_key = "your_twitter_consumer_key_here"
consumer_secret = "your_twitter_consumer_secret_here"
oauth_token = "your_oauth_token_here"
oauth_token_secret = "your_oauth_token_secret"

# Confiruationses
Twitter.configure do |config|
  config.consumer_key = consumer_key
  config.consumer_secret = consumer_secret
  config.oauth_token = oauth_token
  config.oauth_token_secret = oauth_token_secret

TweetStream.configure do |config|
  config.consumer_key = consumer_key
  config.consumer_secret = consumer_secret
  config.oauth_token = oauth_token
  config.oauth_token_secret = oauth_token_secret
  config.auth_method = :oauth
  config.parser = :yajl

# Let's make us a bounding box to give Twitter's streaming API
N = @from_lat + radius
S = @from_lat - radius
E = @from_lng + radius
W = @from_lng - radius

def parse_tweet status
  return if excluded_users.include? status[:user][:screen_name] 

  if status[:coordinates] and status[:coordinates][:type] == 'Point'
    lng, lat = status[:coordinates][:coordinates]

    if    lng < [E, W].max \
      and lng > [E, W].min \
      and lat < [N, S].max \
      and lat > [N, S].min

      @logger.info "Got one! Replying to @#{status[:user][:screen_name]}:"
      @logger.info "\t#{status[:id]}: \"#{status[:text]}\""

      if not status[:in_reply_to_user_id] \
        and not status[:retweeted] \
        and status[:entities][:user_mentions].empty? \
        and not hit_recently(status[:user][:id])

        tweet = Twitter.update(
          "@#{status[:user][:screen_name]} #{random_phrase}",
          :in_reply_to_status_id => status[:id],
          :lat => @from_lat,
          :long => @from_lng,
          :display_coordinates => true

        @user_cache[status[:user][:id]] = Time.now

        @logger.info "\t#{tweet[:id]}: \"#{tweet[:text]}\""
        @logger.info "Didn't reply - tweet was mention, retweet, reply, or spammy."
        @logger.info "In reply to: " + status[:in_reply_to_user_id].inspect
        @logger.info "Retweeted? " + status[:retweeted].inspect
        @logger.info "Mentioned: " + status[:entities][:user_mentions].inspect
        @logger.info "User last hit at: " + @user_cache[status[:user][:id]].inspect

      km_away = Math.sqrt(((lat - @from_lat) * 111)**2 + ((lng - @from_lng) * 79)**2)
      @logger.info "Tweet not within bounding box:\t#{km_away} km away."
rescue Exception => ex
  @logger.error ex.message
  @logger.error ex.backtrace.join "\n"

client = TweetStream::Daemon.new('preacher', :log_output => true)
client.on_error { |message| @logger.error message }
client.on_reconnect { |timeout, retries| @logger.error "Reconnect: timeout = #{timeout}, retries = #{retries}" }

# Start filtering based on location
  @logger.info "Starting up the Street Preacher..."
  client.locations("#{W},#{S},#{E},#{N}") { |status| parse_tweet status }
rescue HTTP::Parser::Error => ex
  # Although TweetStream should recover from
  # disconnections, it fails to do so properly.
  @logger.error "HTTP Parser error encountered - let's sleep for #{ERROR_TIMEOUT}s."
  @logger.error ex.message
  @logger.error ex.backtrace.join "\n"

Feel free to fork it, repurpose it, and do whatever! (Just keep my name at the top, if you please.)


Now read this

Dangerously Convenient APIs

The modern trend of providing an API for everything is wonderful. With minimal effort, any developer with an internet connection can programmatically access a wealth of data and powerful functionality. Without APIs, many hackathons... Continue →