In this lab we are going to get comfortable coding in Javascript and learn how to create a backend RESTful web service that can talk to a database.

By the end of this lab, you should have a basic understanding of how to build a simple 3-tier web application consisting of these three layers:

• Frontend (or client/browser)
• Backend (or server)
• Storage (or database)

The stack we are going to use is called MERN. MERN stands for MongoDB, Express, React, NodeJS.

• MongoDB — document database
• ExpressJS — NodeJS web framework
• ReactJS — a client-side JavaScript framework
• NodeJS — JavaScript web server

The previous lab taught you how to use Javascript and frameworks such as React to build the Client. This lab will focus on how to build a backend service and how to connect it to a database.

We are mainly going to be using technologies from the Javascript ecosystem, so in our case we will be using React JS for the frontend, Node JS + Express for the backend, and Mongo DB for the storage.

If you have not done so already, please install Node, NPM, and a code editor like VS Code or Atom by following the instructions on their websites.

NodeJS is the Javascript runtime that allows us to run Javascript on the server. Any server code you will write in Javascript will run on top of Node so it's a good idea to get a basic understanding of node before attempting to build anything with it.

Go through the first 6 sections of this tutorial:

$ npm install -g learnyounode # use sudo if you run into permission issues
$ learnyounode

Express JS is a server framework that allows you to quickly build a RESTful backend on top of Node JS.

Let's write a simple hello world service using Express. Create a new server directory and follow the next steps:

• Use

   npm init 

• Install express using

  npm install --save express

• Create a new index.js file inside your server directory and paste the following code inside:

const express = require('express')
const bodyParser = require('body-parser')
const app = express()
const mongodb = require('mongodb')
 
 
// Connect to MongoDB database using a MongoDB client for Node
let db
mongodb.MongoClient.connect('mongodb://localhost:27017/', (err, client) => {
    if (err) return console.log(err)
    console.log('Connected to database')
    db = client.db('university-db');
})
 
//Returns middleware that only parses json
app.use(bodyParser.json())
 
//Returns middleware that only parses urlencoded bodies
//The extended option allows us to choose to parse the URL-encoded data with the qs library (when true).
app.use(bodyParser.urlencoded({ extended: true }))
 
 
//Setting up a new /api/tasks endpoint
app.get('/api/tasks', (req, res) => {
    //Find all documents in the tasks collection and returns them as an array.
    db.collection('tasks').find().toArray((err, result) => {
        if (err) return console.log(err)
        console.log(result);
        res.send({
            tasks: result
        })
    })
})
 
//Server will listen on port 3001
app.listen(3001, (err) => {
    if (err) throw err
    console.log('> Ready on http://localhost:3001')
});

• Update package.json script:

"scripts": {
    "dev": "node index.js",
 },

• Start the application using npm run dev
• By accessing localhost:3001/api/tasks we will get the following response

{"tasks":[]}

In Express, route parameters are essentially variables derived from named sections of the URL. Express captures the value in the named section and stores it in the req.params property. For example, if we want to delete a task after its name, we can setup a /api/tasks/{name} endpoint like this in Express:

app.delete('/api/tasks/:name', (req, res) => {
    console.log(req.params['name']);
    // delete task
})

Now we can delete tasks by sending DELETE requests using Postman or any other REST client to http://localhost:3001/api/tasks/taskName where taskName can be anything we want.

Now that you have a functioning backend service, let's do something more interesting. It's generally a good idea to keep your application's data in a database. This allows us to make the data persistent between multiple user sessions, as well as providing an efficient way to store and read the data. In this lab we are going to make our Node/Express app talk to a Mongo DB Database, but the basic principle is the same regardless of what database you are using.

SQL databases are table-based, while NoSQL databases are document, key-value, graph, or wide-column stores. Some examples of SQL databases include MySQL, Oracle, PostgreSQL, and Microsoft SQL Server. NoSQL database examples include MongoDB, BigTable, Redis, RavenDB Cassandra, HBase, Neo4j, and CouchDB.

SQL databases are vertically scalable in most situations. You’re able to increase the load on a single server by adding more CPU, RAM, or SSD capacity. NoSQL databases are horizontally scalable. You’re able to handle higher traffic by sharding, which adds more servers to your NoSQL database. Horizontal scaling has a greater overall capacity than vertical scaling, making NoSQL databases the preferred choice for large and frequently changing data sets.

In order to start working with Mongo, it's useful to first understand a few key concepts related to how Document Oriented Storage works.

• Database - A database is a physical container for collections. Each database gets its own set of files on the file system. A single MongoDB server typically has multiple databases

• Collection - A collection is a group of MongoDB documents. It is the equivalent of an RDBMS table. A collection exists within a single database. Collections do not enforce a schema. Documents within a collection can have different fields. Typically, all documents in a collection are of similar or related purpose.

• Document - A document is a set of key-value pairs. Documents have dynamic schema. Dynamic schema means that documents in the same collection do not need to have the same set of fields or structure, and common fields in a collection's documents may hold different types of data. The following example shows the document structure of a blog site, which is simply a JSON document:

• _id - An id is a 12 bytes hexadecimal number which assures the uniqueness of every document. You can provide _id while inserting the document. If you don’t provide then MongoDB provides a unique id for every document. These 12 bytes can be broken up like this: first 4 bytes for the current timestamp, next 3 bytes for machine id, next 2 bytes for process id of MongoDB server and remaining 3 bytes are simple incremental VALUE.

• Field - A field is a key-value pair in a document. A document has zero or more fields. Fields are analogous to columns in a relational database.

• Cursor - The cursor is a pointer to the result set of a query. Clients can iterate through a cursor to retrieve results.

For our database we are going to use a cloud MongoDB service called Atlas. In the next section we are going to create our own MongoDB instance in the cloud.

1. Visit Atlas and create an account
2. Create a new database by clicking “Build a Database”
3. Next we are gonna choose the free tier
4. Select our region (Frankfurt) and provider (AWS) and click on “Create Cluster”

1. Create a new authentication user. This user will be used to connect to our database. Please remember the password as it will be necessary to connect to our database.

1. Select “Local Environment” and add 0.0.0.0/0 to the access list. The 0.0.0.0/0 will match every IP address.

1. After our Cluster is created go to Browse Collections → Add my own data
2. Select “university-db” as database name and “tasks” as collection name and click Create
3. After our database is created, click on Insert Document (right side of the screen) and insert a new document with a name field. We should see a new document inserted in our database

Great, now we have a Mongo database with our tasks collection. Let's get our express server to pull the data from the database instead of just having it return some hard coded data.

In our express app directory, let's first install a Mongo DB client library from npm:

$ cd /path/to/my-express-server # the same one we created in the Express section before
$ npm install --save mongodb
$ atom index.js
[...edit index.js so that our server pulls the courses list from the database ...]

Inside /server/index.js let's initialise our MongoDB client

const mongodb = require('mongodb')
 
let db   //our db instance
mongodb.MongoClient.connect('mongodb://localhost:27017/', (err, client) => {
    // connecting to our local database using the MongoDB client for Node
    if (err) return console.log(err)
    console.log('Connected to database')
    // get our db instance
    db = client.db('university-db');
})

In order to connect to our database we need a connection string. Go back to the Atlas Cluster and click on Connect → Connect your application → Copy the connection string (don't forget to change the password). The connection string is defined as a connection format to join the MongoDB database server, we are using the username, hostname, password, and port parameter to connect to our database server.

Now all we need to do is update our connection string in our code. For user and password we are going to use the ones we created earlier.

const mongodb = require('mongodb')
 
let db
mongodb.MongoClient.connect('mongodb+srv://<user>:<parola>@cluster......mongodb.net/?retryWrites=true&w=majority', (err, client) => {
    if (err) return console.log(err)
    console.log('Connected to database')
    db = client.db('university-db');
})

Now that we initialised our MongoDB client we can retrieve the documents from our database:

server.get('/api/tasks', (req, res) => {
    db.collection('tasks').find().toArray((err, result) => {
        if (err) return console.log(err)
        res.send({
            tasks: result
        })
    })
})

To check that it's working, send a GET request to http://localhost:3001/api/tasks using Postman (or access the link using your browser), you should be able to retrieve the document that you inserted earlier. Try adding one more document to the database and then refresh the browser.

We are going to take advantage of the MongoDB client available for Node to write our queries.

To retrieve documents in Mongo we are gonna use the find command.To get all documents in a collection we can call:

db.collection('tasks').find().toArray((err, result) => {
    if (err) return console.log(err)
    //do something with result
    console.log(result)
})

If we want to get a single document we can do it using its id (or any other field), the next code snippet will find the document with id 123:

db.collection('tasks').findOne({"_id": 123}).then((err, result) => {
    if (err) return console.log(err)
    //do something with result
    console.log(result)
})

Mongo supports a wide variety of operators such as $gt (greater than), $lt (lower than), $eq, $in etc. Let's write a mongo query that finds all the documents that have the qty field greater than 4:

db.collection('tasks').find({ qty: { $gt: 4 } }).toArray((err, result) => {
    if (err) return console.log(err)
    //do something with result
    console.log(result)
})

If we want to modify a document, we need to use the update method alongside the $set operator. The next query will set the title and info.description fields of the document with id 1 to new values.

db.collection('tasks').updateOne(
    { _id: 1 },
    { $set: {
       title: "ABC123",
       "info.description": "to do...", 
      }
    }
)

In order to delete a document we can use the delete command. This query will delete only one document (the first document that it finds) that matches the title with “ABC123”.

db.collection('tasks').deleteOne(
    { title: "ABC123" }
)

Cross-Origin Resource Sharing (CORS) is an HTTP-header based mechanism that allows a server to indicate any origins (domain, scheme, or port) other than its own from which a browser should permit loading resources. In order for our backend to accept requests from our frontend server (or any other origin) we need to configure CORS.

In our server/index.js paste the following code snippet:

const cors = require('cors')
 
const corsOptions = {
    headers: [
        { key: "Access-Control-Allow-Credentials", value: "true" },
        { key: "Access-Control-Allow-Origin", value: "*" },
        // ...
    ],
    origin: "*", // accept requests from any hostname
    optionsSuccessStatus: 200, // some legacy browsers (IE11, various SmartTVs) choke on 204
};
app.use(cors()) // this must be called at the **top** of our file

Now, when we configure our endpoints we can do it like this:

app.post('/api/tasks', cors(corsOptions), (req, res) => {
    //do something
})

As you know from our SSR lab, NextJS will pre-render the page on each request using the data returned by getServerSideProps(). But what if we want to change our data? Let's say we have a list of items that is pre-rendered, if we want to add a new item to it, the changes will only be visible after we reload the page. If we want the changes to be visible right away we can do a nifty trick to solve our problem.

import { useRouter } from 'next/router';
 
function SomePage(props) {
  const router = useRouter();
 
  // Call this function whenever you want to
  // refresh props!
  const refreshData = () => {
    router.replace(router.asPath);
  }
}
export async function getServerSideProps(context) {
  // Database logic here
}

The refreshData function would be called whenever you want to pull new data from the backend. It'll vary based on your usecase.

But why does it work? Our solution works because we're performing a client-side transition to the same route. router.asPath is a reference to the current path. If we're on /tasks, we're telling Next to do a client-side redirect to /tasks, which causes it to re-fetch the data as JSON, and pass it to the current page as props.

router.replace is like router.push, but it doesn't add an item to the history stack. We don't want this to “count” as a redirect, so that the browser's “Back” button still works as we intend.

You now have a fully functioning backend service that can talk to a database. Using Next, Express and Mongo, create a todo list that fetches, displays and add tasks to the list.

• Download the project archive lab8 and run yarn install and npm run dev.
• Create a new /server directory (outside of our lab8 directory) and install express
• Add a new /server/index.js file and initialize the server as shown in the documentation
• Create a new Cloud MongoDB database (if you haven't already) and insert a few new documents in the tasks collection
• Create a new /api/tasks endpoint that retrieves all the tasks using the Mongo client for NodeJS as shown in the documentation
• Update the TasksList component to retrieve the tasks from our database using getServerSideProps() Hint: Follow the TODOs
• Add the ability to create and delete tasks Hint: Follow the TODOs
• Bonus: Add the ability to edit tasks

Please take a minute to fill in the feedback form for this lab.