Collecting Data with the Twitter API V2

In this section, we explore how to use the Twitter API V2 to collect data from Twitter. This includes the full process, from creating a Twitter developer account to setting up a Google Colab environment for running API queries and retrieving data in real time.

Step 1: Create a Twitter Developer Account

To use the Twitter API, you first need a Twitter developer account:

  1. Sign up: go to Twitter Developer and sign in with your Twitter account.
  2. Request developer access: click Apply for a Developer Account and follow the instructions. This includes describing your project and explaining why you want API access.
  3. Confirmation and access: once your request is approved, you will gain access to the Twitter developer portal.

V2 API Pricing

Since the recent changes under Elon Musk, access to the Twitter API has become paid, with three main tiers:

  1. Free plan: only allows tweet publication, up to 1,500 tweets per month. It does not allow extraction of user tweets.
  2. Basic plan: priced at $100 per month, it allows extraction of up to 10,000 tweets per month with broader interaction limits.
  3. Enterprise plan: offers large-scale access with custom pricing.

These options are designed to monetize the API and restrict access to data.

More details.

Step 2: Create a Project and an Application

After your account is approved:

  1. Create a project: in the developer dashboard, create a new project by clicking Create Project. Give it a name and a description that fit your objectives.
  2. Generate API keys: when creating the associated application, Twitter will provide API keys and security tokens. Keep them carefully because they will be required for all your API requests.

Step 3: Set Up a Google Colab Environment

To run your API requests in Python, Google Colab provides a convenient environment:

Install the Required Libraries

In Google Colab, run the following command to install tweepy, a Python library for interacting with the Twitter API.

!pip install tweepy

Configure the API Keys

Add your API keys and security tokens to your Colab code:

import tweepy
# Add your keys
api_key = 'YOUR_API_KEY'
api_key_secret = 'YOUR_API_SECRET'
access_token = 'YOUR_ACCESS_TOKEN'
access_token_secret = 'YOUR_ACCESS_TOKEN_SECRET'
# Authentication
auth = tweepy.OAuthHandler(api_key, api_key_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)

Retrieve Data with the API

Example of a tweet search:

for tweet in tweepy.Cursor(api.search_tweets, q="your keyword", lang="fr").items(100):
    print(tweet.text)

This code extracts tweets containing a given keyword and displays them in the console.

Analyze the Results

The extracted data can then be analyzed with text-mining techniques such as sentiment analysis or topic modeling.