Javascript is required
·
5 min read

Track Twitter Follower Growth Over Time Using A Serverless Node.js API on AWS Amplify

Track Twitter Follower Growth Over Time Using A Serverless Node.js API on AWS Amplify Image

In March 2021 I started to use FeedHive to help me grow an audience on Twitter. Recently, I wanted to check how my Twitter followers have grown over time. Unfortunately, Twitter Analytics only provides data from the last 30 days. So I decided to develop a simple serverless API to fetch and store my follower count each month.

Tech Stack

As I already use AWS Amplify for some private APIs, I wanted to reuse this framework for this new project.

AWS Amplify Architecture

For this new project I need the following components:

  • React for the frontend web application which will fetch the data from my serverless API
  • AWS API Gateway which provides traffic management, CORS support, authorization and access control, throttling, monitoring, and API version management for the new API
  • AWS Lambda with Node.js that fetches the follower count from Twitter API
  • AWS DynamoDB which is a NoSQL database and which will store the follower count

Fetching follower count from backend

The first step is to add a new Node.js REST API to our Amplify application that provides a /twitter endpoint which is triggered on a recurring schedule. In my case, it will be on every 1st day of the month. The official documentation will help you to set up such a new REST API.

To be able to fetch the follower count from Twitter API I decided to use FeedHive's Twitter Client. This library needs four secrets to be able to access Twitter API. We will store them in the AWS Secret Manager, my article "How to Use Environment Variables to Store Secrets in AWS Amplify Backend" will guide you through this process.

After the API is created and pushed to the cloud, we can write the basic functionality to fetch the Twitter followers inside our AWS Lambda function:

1const twitterApiClient = require('twitter-api-client')
2const AWS = require('aws-sdk')
3
4const twitterUsername = 'yourTwitterUsername'
5const secretsManager = new AWS.SecretsManager()
6const responseHeaders = {
7  'Content-Type': 'application/json',
8  'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token',
9  'Access-Control-Allow-Methods': 'OPTIONS,POST',
10  'Access-Control-Allow-Credentials': true,
11  'Access-Control-Allow-Origin': '*',
12  'X-Requested-With': '*',
13}
14
15exports.handler = async (event) => {
16  const secretData = await secretsManager.getSecretValue({ SecretId: 'prod/twitterApi/twitter' }).promise()
17  const secretValues = JSON.parse(secretData.SecretString)
18
19  const twitterClient = new twitterApiClient.TwitterClient({
20    apiKey: secretValues.TWITTER_API_KEY,
21    apiSecret: secretValues.TWITTER_API_KEY_SECRET,
22    accessToken: secretValues.TWITTER_ACCESS_TOKEN,
23    accessTokenSecret: secretValues.TWITTER_ACCESS_TOKEN_SECRET,
24  })
25
26  try {
27    const response = await twitterClient.accountsAndUsers.usersSearch({
28      q: twitterUsername,
29    })
30    const followersCount = response[0].followers_count
31
32    return {
33      statusCode: 200,
34      headers: responseHeaders,
35      body: followersCount,
36    }
37  } catch (e) {
38    console.error('Error:', e)
39    return {
40      statusCode: 500,
41      headers: responseHeaders,
42      body: e.message ? e.message : JSON.stringify(e),
43    }
44  }
45}

The next step is to add DynamoDB support to be able to store a new follower count and get a list of the stored data.

Therefore, we need to add a new storage to our AWS Amplify application, see "Adding a NoSQL database" for detailed instructions.

We are adding a NoSQL table that has the following columns:

  • id: A unique string identifier for each row as a string
  • follower_count: the current follower count as number
  • data: an ISO timestamp string that represents the time when the follower count was fetched

Now, we need to allow our Lambda function to access this storage:

bash
1 amplify update function
2? Select the Lambda function you want to update twitterfunction
3? Which setting do you want to update? Resource access permissions
4? Select the categories you want this function to have access to. storage
5? Storage has 3 resources in this project. Select the one you would like your Lambda to access twitterdynamo
6? Select the operations you want to permit on twitterdynamo create, read, update, delete

Finally, we can use the AWS SDK to store and read from DynamoDB:

1const twitterApiClient = require('twitter-api-client')
2const AWS = require('aws-sdk')
3const { v4: uuidv4 } = require('uuid')
4
5const secretsManager = new AWS.SecretsManager()
6
7const twitterUsername = 'yourTwitterUsername'
8const responseHeaders = {
9  'Access-Control-Allow-Origin': '*',
10  // ...
11}
12
13const docClient = new AWS.DynamoDB.DocumentClient()
14
15let tableName = 'twittertable'
16if (process.env.ENV && process.env.ENV !== 'NONE') {
17  tableName = `${tableName}-${process.env.ENV}`
18}
19
20const tableParams = {
21  TableName: tableName,
22}
23
24async function getStoredFollowers() {
25  console.log(`👷 Start scanning stored follower data...`)
26  return docClient.scan({ ...tableParams }).promise()
27}
28
29async function storeFollowersCount(followerCount) {
30  console.log(`👷 Start storing follower count...`)
31  return docClient
32    .put({
33      ...tableParams,
34      Item: {
35        id: uuidv4(),
36        follower_count: followerCount,
37        date: new Date().toISOString(),
38      },
39    })
40    .promise()
41}
42
43async function fetchFollowerCount(twitterClient) {
44  console.log(`👷 Start fetching follower count...`)
45  const