Javascript is required
·
5 min read
·
2967 views

Track Twitter Follower Growth Over Time Using A Serverless Node.js API on AWS Amplify

Track Twitter Follower Growth Over Time Using A Serverless Node.js API on AWS Amplify Image

In March 2021 I started to use FeedHive to help me grow an audience on Twitter. Recently, I wanted to check how my Twitter followers have grown over time. Unfortunately, Twitter Analytics only provides data from the last 30 days. So I decided to develop a simple serverless API to fetch and store my follower count each month.

Tech Stack

As I already use AWS Amplify for some private APIs, I wanted to reuse this framework for this new project.

AWS Amplify Architecture

For this new project I need the following components:

  • React for the frontend web application which will fetch the data from my serverless API
  • AWS API Gateway which provides traffic management, CORS support, authorization and access control, throttling, monitoring, and API version management for the new API
  • AWS Lambda with Node.js that fetches the follower count from Twitter API
  • AWS DynamoDB which is a NoSQL database and which will store the follower count

Fetching follower count from backend

The first step is to add a new Node.js REST API to our Amplify application that provides a /twitter endpoint which is triggered on a recurring schedule. In my case, it will be on every 1st day of the month. The official documentation will help you to set up such a new REST API.

To be able to fetch the follower count from Twitter API I decided to use FeedHive's Twitter Client. This library needs four secrets to be able to access Twitter API. We will store them in the AWS Secret Manager, my article "How to Use Environment Variables to Store Secrets in AWS Amplify Backend" will guide you through this process.

After the API is created and pushed to the cloud, we can write the basic functionality to fetch the Twitter followers inside our AWS Lambda function:

1const twitterApiClient = require('twitter-api-client')
2const AWS = require('aws-sdk')
3
4const twitterUsername = 'yourTwitterUsername'
5const secretsManager = new AWS.SecretsManager()
6const responseHeaders = {
7  'Content-Type': 'application/json',
8  'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token',
9  'Access-Control-Allow-Methods': 'OPTIONS,POST',
10  'Access-Control-Allow-Credentials': true,
11  'Access-Control-Allow-Origin': '*',
12  'X-Requested-With': '*',
13}
14
15exports.handler = async (event) => {
16  const secretData = await secretsManager.getSecretValue({ SecretId: 'prod/twitterApi/twitter' }).promise()
17  const secretValues = JSON.parse(secretData.SecretString)
18
19  const twitterClient = new twitterApiClient.TwitterClient({
20    apiKey: secretValues.TWITTER_API_KEY,
21    apiSecret: secretValues.TWITTER_API_KEY_SECRET,
22    accessToken: secretValues.TWITTER_ACCESS_TOKEN,
23    accessTokenSecret: secretValues.TWITTER_ACCESS_TOKEN_SECRET,
24  })
25
26  try {
27    const response = await twitterClient.accountsAndUsers.usersSearch({
28      q: twitterUsername,
29    })
30    const followersCount = response[0].followers_count
31
32    return {
33      statusCode: 200,
34      headers: responseHeaders,
35      body: followersCount,
36    }
37  } catch (e) {
38    console.error('Error:', e)
39    return {
40      statusCode: 500,
41      headers: responseHeaders,
42      body: e.message ? e.message : JSON.stringify(e),
43    }
44  }
45}

The next step is to add DynamoDB support to be able to store a new follower count and get a list of the stored data.

Therefore, we need to add a new storage to our AWS Amplify application, see "Adding a NoSQL database" for detailed instructions.

We are adding a NoSQL table that has the following columns:

  • id: A unique string identifier for each row as a string
  • follower_count: the current follower count as number
  • data: an ISO timestamp string that represents the time when the follower count was fetched

Now, we need to allow our Lambda function to access this storage:

bash
1 amplify update function
2? Select the Lambda function you want to update twitterfunction
3? Which setting do you want to update? Resource access permissions
4? Select the categories you want this function to have access to. storage
5? Storage has 3 resources in this project. Select the one you would like your Lambda to access twitterdynamo
6? Select the operations you want to permit on twitterdynamo create, read, update, delete

Finally, we can use the AWS SDK to store and read from DynamoDB:

1const twitterApiClient = require('twitter-api-client')
2const AWS = require('aws-sdk')
3const { v4: uuidv4 } = require('uuid')
4
5const secretsManager = new AWS.SecretsManager()
6
7const twitterUsername = 'yourTwitterUsername'
8const responseHeaders = {
9  'Access-Control-Allow-Origin': '*',
10  // ...
11}
12
13const docClient = new AWS.DynamoDB.DocumentClient()
14
15let tableName = 'twittertable'
16if (process.env.ENV && process.env.ENV !== 'NONE') {
17  tableName = `${tableName}-${process.env.ENV}`
18}
19
20const tableParams = {
21  TableName: tableName,
22}
23
24async function getStoredFollowers() {
25  console.log(`👷 Start scanning stored follower data...`)
26  return docClient.scan({ ...tableParams }).promise()
27}
28
29async function storeFollowersCount(followerCount) {
30  console.log(`👷 Start storing follower count...`)
31  return docClient
32    .put({
33      ...tableParams,
34      Item: {
35        id: uuidv4(),
36        follower_count: followerCount,
37        date: new Date().toISOString(),
38      },
39    })
40    .promise()
41}
42
43async function fetchFollowerCount(twitterClient) {
44  console.log(`👷 Start fetching follower count...`)
45  const data = await twitterClient.accountsAndUsers.usersSearch({
46    q: twitterUsername,
47  })
48  return data[0].followers_count
49}
50
51exports.handler = async (event) => {
52  const secretData = await secretsManager.getSecretValue({ SecretId: 'prod/twitterApi/twitter' }).promise()
53  const secretValues = JSON.parse(secretData.SecretString)
54
55  const twitterClient = new twitterApiClient.TwitterClient({
56    apiKey: secretValues.TWITTER_API_KEY,
57    apiSecret: secretValues.TWITTER_API_KEY_SECRET,
58    accessToken: secretValues.TWITTER_ACCESS_TOKEN,
59    accessTokenSecret: secretValues.TWITTER_ACCESS_TOKEN_SECRET,
60  })
61
62  try {
63    const followersCount = await fetchFollowerCount(twitterClient)
64
65    await storeFollowersCount(followersCount)
66    const storedFollowers = await getStoredFollowers()
67
68    return {
69      statusCode: 200,
70      headers: responseHeaders,
71      body: JSON.stringify(storedFollowers.Items),
72    }
73  } catch (e) {
74    console.error('Error:', e)
75    return {
76      statusCode: 500,
77      headers: responseHeaders,
78      body: e.message ? e.message : JSON.stringify(e),
79    }
80  }
81}

A successful API response will have a similar JSON array in its body:

json
1[
2  {
3    "follower_count": 350,
4    "date": "2021-08-09T11:39:50.885Z",
5    "id": "9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d"
6  },
7  {
8    "follower_count": 380,
9    "date": "2021-09-09T11:39:50.885Z",
10    "id": "a5a2a894-166b-4672-aefe-cea01c70a01a"
11  }
12]

Show data in frontend

To be able to show the data in the React frontend I use the Recharts library which is "a composable charting library built on React components".

The React component is quite simple and uses the AWS Amplify REST API library to fetch the data from our API endpoint:

1import { API } from 'aws-amplify';
2import { useState } from 'react';
3import {
4  LineChart,
5  Line,
6  XAxis,
7  YAxis,
8  CartesianGrid,
9  Tooltip,
10  Legend,
11} from 'recharts';
12
13const TwitterPage = () => {
14  const [isLoading, setIsLoading] = useState(false);
15  const [apiError, setApiError] = useState();
16  const [followerData, setFollowerData] = useState();
17
18  const triggerEndpoint = async () => {
19    setIsLoading(true);
20
21    try {
22      const data = await API.get('twitterapi', '/twitter');
23      setFollowerData(
24        data.map((d) => {
25          d.followers = d.follower_count;
26          d.date = new Date(d.date).toLocaleDateString();
27          return d;
28        })
29      );
30    } catch (error) {
31      console.error('Failed to trigger Twitter endpoint', error);
32      setApiError(JSON.stringify(error));
33    } finally {
34      setIsLoading(false);
35    }
36  };
37
38  return (
39    <div>
40      <h1>Twitter API</h1>
41      <section>
42        <button onClick={triggerEndpoint}>
43          Fetch followers
44        </button>
45        {followerData ? (
46          <LineChart
47            width={700}
48            height={500}
49            data={followerData}
50            margin={{
51              top: 30,
52              bottom: 30,
53            }}
54          >
55            <CartesianGrid strokeDasharray="3 3" />
56            <XAxis dataKey="date" />
57            <YAxis dataKey="followers" />
58            <Tooltip />
59            <Legend />
60            <Line
61              type="monotone"
62              dataKey="followers"
63              stroke="#FC1A20"
64              activeDot={{ r: 8 }}
65            />
66          </LineChart>
67        ) : null}
68        {apiError ? <p className="py-4">{JSON.parse(apiError)}</p> : null}
69      </section>
70    </Layout>
71  );
72};
73
74export default TwitterPage;

which results in such a graph:

Twitter Follower Count Graph

Conclusion

Using serverless functions it is quite easy and cheap to build a custom solution to track Twitter follower growth over time.

What do you use to track your follower growth? Leave a comment and tell me about your solution.

If you liked this article, follow me on Twitter to get notified about new blog posts and more content from me.

I will never share any of your personal data. You can unsubscribe at any time.

If you found this article helpful.You will love these ones as well.
How I Replaced Revue With a Custom-Built Newsletter Service Using Nuxt 3, Supabase, Serverless, and Amazon SES Image

How I Replaced Revue With a Custom-Built Newsletter Service Using Nuxt 3, Supabase, Serverless, and Amazon SES

Build and Deploy a Serverless GraphQL React App Using AWS Amplify Image

Build and Deploy a Serverless GraphQL React App Using AWS Amplify

The 10 Favorite Features of My Developer Portfolio Website Image

The 10 Favorite Features of My Developer Portfolio Website

How I Built a Twitter Keyword Monitoring Using a Serverless Node.js Function With AWS Amplify Image

How I Built a Twitter Keyword Monitoring Using a Serverless Node.js Function With AWS Amplify