Build a news chatbot using Next.js and Dialogflow

  • Christian Nwamba
May 26th, 2018
You will need Node and npm or Yarn installed on your machine. Some knowledge of JavaScript will be useful.

Machine learning, natural language process and AI, in general, has gained more audience with the massive growth and investment it has received. Computers can now easily understand and derive meaning from human language. This has opened a new window of opportunity for application developers to build new kinds of products that will interact differently with individuals.

Today, we’ll be building a simple chatbot that can help search for news from the internet by talking to it as you would a human being. We’ll call this “Channel 24 news with Bobby Mc-newsfeed”.

Prerequisites

Kindly ensure you have Node, Npm or Yarn installed on your machine before moving past this section. This will be needed for running and managing the dependencies needed by our application. Also, no knowledge of React is required, but a basic understanding of JavaScript may be helpful.

  1. Next.js: this is a framework for developing server-side rendered applications, just as you would with PHP, but this time with React.
  2. Pusher: this is a framework that allows you to build realtime applications with its easy to use pub/sub messaging API.
  3. Dialogflow: An end-to-end development suite for building conversational interfaces for websites, mobile applications, popular messaging platforms, and IoT devices
  4. NewsApi: A simple API for getting realtime news from the web.

App structure

We'll start by setting up our Next.js application. The easiest way to go about this is to clone the nextjs-javascript-starter repo. Run:

    git clone https://github.com/Robophil/nextjs-javascript-starter.git chat-bot

This will clone the starter pack into a directory called chat-bot. Our app directory will look like this.

  1. components: any Next.js component we'll create will go here.
  2. css: styles for our components and pages would go here.
  3. pages: any .js file in this directory would be served as a page. So any page we'll want to create would go here.

Install dependencies

To install the dependencies needed by our app, run:

    # enter app directory
    cd chat-bot

    # install dependencies from nextjs-javascript-starter
    yarn install

    # add client-side dependencies
    yarn add axios pusher-js react-chat-ui

    # add server-side dependencies
    yarn add cors express pusher body-parser dialogflow country-data newsapi dlv

Now we have all dependencies needed by our app installed.

Getting our Pusher credentials

If you don't have a Pusher account already, kindly create one here. Once you have an account, simply head down to your dashboard and create an app. Once that is done, click on App Keys and note your credentials. We'll be needing them in a bit.

Setting up Dialogflow

Dialogflow is responsible for analyzing the messages sent to Bobby Mc-newsfeed and breaking them down into meaningful intents that can be processed by our new API. Simply head down here and create an account. Dialogflow already comes with some prebuilt agents that might cover your application needs. Simply click on "Prebuilt Agent" by the left and import the News agent. You can easily search for it to filter out the other available agents on the list.

This would create a new agent for you called News. Our new agent News comes with some already trained intent and entities called sort, source and topic. These are the entities we’ll want to extract from each message sent down to our agent.

  • sort: tells us if the message contains a request for headlines or top stories
  • source: tells us the source if specified, such as BBC or CNN
  • topic: tells us the category such as business, sports, health

The agent also makes use of some prebuilt entities like geo-location which tells us the country specified.

Click on the setting icon beside your project name, to see more information about the agent you’ve created. You might see a notice about API migration from V1 to V2, like this:

NOTE: This project uses the V2 API, so ensure the API version is set to V2.

Next, we’ll need to get credentials to run this locally. Head down to google cloud console and enable this project.

  1. Select the project from the drop-down beside the header “Google Cloud Platform”. This would be called “News”.

  1. Click on the dashboard on the left then ENABLE APIS AND SERVICES

Simply search for “Dialogflow” and click Enable API. To get our credentials, click on Credentials, then the drop-down Create Credentials and then service account key. Select JSON as your format and begin the download.

Keep this file close by as you’ll be needing it soon. More detailed instruction here.

News API credentials

Head over to the News API homepage and sign up. You’ll get a key needed in the coming section. Now that we have all our credentials ready, we can get on with building our application.

Creating the index page

Any .js file created in the pages directory becomes a page that can be served by Next.js. We already have an index.js file in our page directory. Replace the content with the following below:

    // pages/index.js
    import React from 'react'
    import axios from 'axios'
    import { ChatFeed, Message } from 'react-chat-ui'
    import Pusher from 'pusher-js'
    import MyChatBubble from '../components/MyChatBubble'
    import '../css/chat.css'

    const pusher = new Pusher('key', {
      cluster: 'eu',
      encrypted: true
    })

    const channel = pusher.subscribe('news')

    export default class Index extends React.Component {
      constructor (props) {
        super(props)
        this.state = {
          message: '',
          isTyping: false,
          messages: [
            new Message({
              id: 'bot',
              message: 'hello',
              senderName: 'bot'
            }),
            new Message({
              id: 2,
              message: 'hi bobby'
            })
          ]}
      }

      componentDidMount () {
        this.receiveUpdateFromPusher()
      }

      /**
      * When there's an update,
      * append the message to the Chat field.
      */
      receiveUpdateFromPusher () {
        channel.bind('news-update', articles => {
          const messages = articles.map(article => {
            return new Message({
              id: 'bot',
              message: `
                Description: ${article.title || article.description}
                Url: ${article.url}
              `
            })
          })
          const intro = messages.length ? 'Here you go, i found some for you' : 'I couldn\'t find any :-(. Search for something else'
          this.setState({
            messages: [
              ...this.state.messages,
              new Message({
                id: 'bot',
                message: intro
              }),
              ...messages
            ]
          })
        })
      }

      handleChange (event) {
        this.setState({message: event.target.value})
      }

      /**
      * When user press Enter, send message to the server
      */
      handleKeyPress (e) {
        if (e.key === 'Enter' && e.target.value) {
          const message = e.target.value
          this.setState({message: ''})
          this.pushMessage(message)

          axios.post('http://localhost:8080/message', {message})
          .then(res => {
            console.log('received by server')
          })
          .catch(error => {
            throw error
          })
        }
      }

      pushMessage (message) {
        this.setState({
          messages: [
            ...this.state.messages,
            new Message({
              id: 2,
              message
            })
          ]
        })
      }

      render () {
        return (
          <div className='chat-wrapper'>
            <h3> Channel 24: News worldwide with Bobby Mc-newsfeed</h3>
            <ChatFeed
              messages={this.state.messages} // Boolean: list of message objects
              isTyping={this.state.isTyping} // Boolean: is the recipient typing
              hasInputField={false} // Boolean: use our input, or use your own
              showSenderName // show the name of the user who sent the message
              bubblesCentered={false} // Boolean should the bubbles be centered in the feed?
              chatBubble={MyChatBubble}
              maxHeight={750}
            />
            <input className='chat-input' type='text' value={this.state.message}
              onChange={this.handleChange.bind(this)}
              onKeyPress={this.handleKeyPress.bind(this)}
              placeholder='Type a message...' />
          </div>
        )
      }
    }

Replace the key placeholder string in the Pusher configuration above with your Pusher key.

ChatBubble component

Since we’re building a chatbot, we’ll be needing to represent the message in some fancy bubble as we’re used to seeing. We’ll be using react-chat-ui to create this, but with a custom chat bubble, which styles our bubble differently depending on the message sender.

    # To create file, run:
    touch components/MyChatBubble.js

Add the following code to the file:

    // components/MyChatBubble.js
    import React from 'react'
    import { ChatBubble } from 'react-chat-ui'

    export default class MyChatBubble extends React.Component {
      render () {
        let bubbleStyles = {}
        if (this.props.message.id === 'bot') {
          bubbleStyles = {
            text: {
              fontSize: 12
            },
            chatbubble: {
              borderRadius: 70,
              padding: '10px 30px'
            }
          }
        } else {
          bubbleStyles = {
            text: {
              fontSize: 12
            },
            chatbubble: {
              borderRadius: 70,
              padding: '10px 30px',
              backgroundColor: 'black',
              float: 'right'
            }
          }
        }
        return <ChatBubble bubbleStyles={bubbleStyles} message={this.props.message} />
      }
    }


## ChatBubble style
    # To create file, run:
    touch css/chat.css

Add the following style to the file:

    /* css/chat.css */
    div#__next {
      display: flex;
      justify-content: center;
    }

    div.chat-wrapper {
      width: 500px;
      border: 1px black solid;
      border-radius: 2%;
      padding: 30px;
    }

    input.chat-input {
      border: none;
      border-top: 1px solid #ddd;
      font-size: 16px;
      padding: 30px;
      width: 90%;
      padding: 20px 10px;
      margin: 5px;
    }

Creating server.js

Our sever.js file is responsible for publishing Pusher events, querying Dialogflow for Intent and communicating with the news API. Messages from the chat come straight to the server on the route /message and forward the message to Dialogflow to interpret and return a result. We can now use the result gotten from Dialogflow to communicate with the news API and get a list of news articles matching the query.

    # To create file, run:
    touch server.js

Add the following code to the file:

    // server.js
    const express = require('express')
    const app = express()
    const bodyParser = require('body-parser')
    const cors = require('cors')
    const Pusher = require('pusher')
    const countryData = require('country-data')
    const dialogflow = require('dialogflow')
    const NewsAPI = require('newsapi')
    const dlv = require('dlv')
    const newsapi = new NewsAPI('enter your news API token')

    const projectId = 'project id' // update with your project Id
    const sessionId = 'a-random-session-id' // use any string as session Id
    const languageCode = 'en-US'

    const sessionClient = new dialogflow.SessionsClient()
    const sessionPath = sessionClient.sessionPath(projectId, sessionId)

    const countryDataByName = countryData.countries.all.reduce((acc, curr) => {
      acc[curr.name.toLowerCase()] = curr
      return acc
    }, {})

    // build query for getting an Intent
    const buildQuery = function (query) {
      return {
        session: sessionPath,
        queryInput: {
          text: {
            text: query,
            languageCode: languageCode
          }
        }
      }
    }

    // use data from intent to  fetch news
    const fetchNews = function (intentData) {
      const category = dlv(intentData, 'category.stringValue')
      const geoCountry = dlv(intentData, 'geo-country.stringValue', '').toLowerCase()
      const country = dlv(countryDataByName, `${geoCountry}.alpha2`, 'us')

      return newsapi.v2.topHeadlines({
        category,
        language: 'en',
        country
      })
    }

    app.use(cors())
    app.use(bodyParser.urlencoded({ extended: true }))
    app.use(bodyParser.json())

    const port = process.env.PORT || 8080

    const pusher = new Pusher({
      appId: 'app_id',
      key: 'key',
      secret: 'secret',
      cluster: 'cluster',
      encrypted: true
    })

    // receive message sent by client
    app.post('/message', function (req, res) {
      return sessionClient
      .detectIntent(buildQuery(req.body.message))
      .then(responses => {
        console.log('Detected intent')
        const result = dlv(responses[0], 'queryResult')
        const intentData = dlv(responses[0], 'queryResult.parameters.fields')

        // if there's a result and an intent
        if (result && result.intent) {
          fetchNews(intentData)
          .then(news => news.articles)
          .then(articles => pusher.trigger('news', 'news-update', articles.splice(0, 6)))
          .then(() => console.log('published to pusher'))
        } else {
          console.log(`  No intent matched.`)
        }
        return res.sendStatus(200)
      })
      .catch(err => {
        console.error('ERROR:', err)
      })
    })

    app.listen(port, function () {
      console.log('Node app is running at localhost:' + port)
    })

Running the app

Copy the service file downloaded from Dialogflow to your project directory. You can now reference this file when starting up the app.

Start the next app with:

    # start next app
    yarn run dev

    # start server app (bash)
    GOOGLE_APPLICATION_CREDENTIALS="service-account-file.json" node server.js

    # in powershell
    $env:GOOGLE_APPLICATION_CREDENTIALS="service-account-file.json"; node server.js

Update GOOGLE_APPLICATION_CREDENTIALS to point to the location of the service account file you downloaded previously to start the server app correctly.

You can ask Bobby Mac-newsfeed simple questions like

  • Show me sports news from Nigeria
  • What’s the latest in business in France?
  • How’s the health sector doing in Canada?

Our agent will try to pick out location, topic and category for your message and use that to query the news API. Feel free to express yourself freely and see what Bobby replies to you with.

Conclusion

We’ve been able to build a chatbot with the ability to send us news updates we requested for. You can extend the functionality of Bobby to handle a lot more and interpret our message more accurately by training our model more.

The repo for this tutorial lives here. Feel free to contribute.

  • Channels

© 2018 Pusher Ltd. All rights reserved.

Pusher Limited is a company registered in England and Wales (No. 07489873) whose registered office is at 160 Old Street, London, EC1V 9BW.