In this lab we are going to get comfortable coding in Javascript and learn how to create a backend RESTful web service that can talk to a database.
By the end of this lab, you should have a basic understanding of how to build a simple 3-tier web application consisting of these three layers:
The stack we are going to use is called MERN. MERN stands for MongoDB, Express, React, NodeJS.
The previous lab taught you how to use Javascript and frameworks such as React to build the Client. This lab will focus on how to build a backend service and how to connect it to a database.
We are mainly going to be using technologies from the Javascript ecosystem, so in our case we will be using React JS for the frontend, Node JS + Express for the backend, and Mongo DB for the storage.
If you have not done so already, please install Node, NPM, and a code editor like VS Code or Atom by following the instructions on their websites.
NodeJS is the Javascript runtime that allows us to run Javascript on the server. Any server code you will write in Javascript will run on top of Node so it's a good idea to get a basic understanding of node before attempting to build anything with it.
Go through the first 6 sections of this tutorial:
$ npm install -g learnyounode # use sudo if you run into permission issues $ learnyounode
Express JS is a server framework that allows you to quickly build a RESTful backend on top of Node JS.
Let's write a simple hello world service using Express. Create a new server directory and follow the next steps:
npm init
npm install --save express
const express = require('express') const bodyParser = require('body-parser') const app = express() const mongodb = require('mongodb') // Connect to MongoDB database using a MongoDB client for Node let db mongodb.MongoClient.connect('mongodb://localhost:27017/', (err, client) => { if (err) return console.log(err) console.log('Connected to database') db = client.db('university-db'); }) //Returns middleware that only parses json app.use(bodyParser.json()) //Returns middleware that only parses urlencoded bodies //The extended option allows us to choose to parse the URL-encoded data with the qs library (when true). app.use(bodyParser.urlencoded({ extended: true })) //Setting up a new /api/tasks endpoint app.get('/api/tasks', (req, res) => { //Find all documents in the tasks collection and returns them as an array. db.collection('tasks').find().toArray((err, result) => { if (err) return console.log(err) console.log(result); res.send({ tasks: result }) }) }) //Server will listen on port 3001 app.listen(3001, (err) => { if (err) throw err console.log('> Ready on http://localhost:3001') });
"scripts": { "dev": "node index.js", },
{"tasks":[]}
In Express, route parameters are essentially variables derived from named sections of the URL. Express captures the value in the named section and stores it in the req.params property. For example, if we want to delete a task after its name, we can setup a /api/tasks/{name} endpoint like this in Express:
app.delete('/api/tasks/:name', (req, res) => { console.log(req.params['name']); // delete task })
Now we can delete tasks by sending DELETE requests using Postman or any other REST client to http://localhost:3001/api/tasks/taskName where taskName can be anything we want.
Now that you have a functioning backend service, let's do something more interesting. It's generally a good idea to keep your application's data in a database. This allows us to make the data persistent between multiple user sessions, as well as providing an efficient way to store and read the data. In this lab we are going to make our Node/Express app talk to a Mongo DB Database, but the basic principle is the same regardless of what database you are using.
SQL databases are table-based, while NoSQL databases are document, key-value, graph, or wide-column stores. Some examples of SQL databases include MySQL, Oracle, PostgreSQL, and Microsoft SQL Server. NoSQL database examples include MongoDB, BigTable, Redis, RavenDB Cassandra, HBase, Neo4j, and CouchDB.
SQL databases are vertically scalable in most situations. You’re able to increase the load on a single server by adding more CPU, RAM, or SSD capacity. NoSQL databases are horizontally scalable. You’re able to handle higher traffic by sharding, which adds more servers to your NoSQL database. Horizontal scaling has a greater overall capacity than vertical scaling, making NoSQL databases the preferred choice for large and frequently changing data sets.
In order to start working with Mongo, it's useful to first understand a few key concepts related to how Document Oriented Storage works.
For our database we are going to use a cloud MongoDB service called Atlas. In the next section we are going to create our own MongoDB instance in the cloud.
Great, now we have a Mongo database with our tasks collection. Let's get our express server to pull the data from the database instead of just having it return some hard coded data.
In our express app directory, let's first install a Mongo DB client library from npm:
$ cd /path/to/my-express-server # the same one we created in the Express section before $ npm install --save mongodb $ atom index.js [...edit index.js so that our server pulls the courses list from the database ...]
Inside /server/index.js let's initialise our MongoDB client
const mongodb = require('mongodb') let db //our db instance mongodb.MongoClient.connect('mongodb://localhost:27017/', (err, client) => { // connecting to our local database using the MongoDB client for Node if (err) return console.log(err) console.log('Connected to database') // get our db instance db = client.db('university-db'); })
In order to connect to our database we need a connection string. Go back to the Atlas Cluster and click on Connect → Connect your application → Copy the connection string (don't forget to change the password). The connection string is defined as a connection format to join the MongoDB database server, we are using the username, hostname, password, and port parameter to connect to our database server.
Now all we need to do is update our connection string in our code. For user and password we are going to use the ones we created earlier.
const mongodb = require('mongodb') let db mongodb.MongoClient.connect('mongodb+srv://<user>:<parola>@cluster......mongodb.net/?retryWrites=true&w=majority', (err, client) => { if (err) return console.log(err) console.log('Connected to database') db = client.db('university-db'); })
Now that we initialised our MongoDB client we can retrieve the documents from our database:
server.get('/api/tasks', (req, res) => { db.collection('tasks').find().toArray((err, result) => { if (err) return console.log(err) res.send({ tasks: result }) }) })
To check that it's working, send a GET request to http://localhost:3001/api/tasks using Postman (or access the link using your browser), you should be able to retrieve the document that you inserted earlier. Try adding one more document to the database and then refresh the browser.
We are going to take advantage of the MongoDB client available for Node to write our queries.
To retrieve documents in Mongo we are gonna use the find command.To get all documents in a collection we can call:
db.collection('tasks').find().toArray((err, result) => { if (err) return console.log(err) //do something with result console.log(result) })
If we want to get a single document we can do it using its id (or any other field), the next code snippet will find the document with id 123:
db.collection('tasks').findOne({"_id": 123}).then((err, result) => { if (err) return console.log(err) //do something with result console.log(result) })
Mongo supports a wide variety of operators such as $gt (greater than), $lt (lower than), $eq, $in etc. Let's write a mongo query that finds all the documents that have the qty field greater than 4:
db.collection('tasks').find({ qty: { $gt: 4 } }).toArray((err, result) => { if (err) return console.log(err) //do something with result console.log(result) })
If we want to modify a document, we need to use the update method alongside the $set operator. The next query will set the title and info.description fields of the document with id 1 to new values.
db.collection('tasks').updateOne( { _id: 1 }, { $set: { title: "ABC123", "info.description": "to do...", } } )
In order to delete a document we can use the delete command. This query will delete only one document (the first document that it finds) that matches the title with “ABC123”.
db.collection('tasks').deleteOne( { title: "ABC123" } )
Cross-Origin Resource Sharing (CORS) is an HTTP-header based mechanism that allows a server to indicate any origins (domain, scheme, or port) other than its own from which a browser should permit loading resources. In order for our backend to accept requests from our frontend server (or any other origin) we need to configure CORS.
In our server/index.js paste the following code snippet:
const cors = require('cors') const corsOptions = { headers: [ { key: "Access-Control-Allow-Credentials", value: "true" }, { key: "Access-Control-Allow-Origin", value: "*" }, // ... ], origin: "*", // accept requests from any hostname optionsSuccessStatus: 200, // some legacy browsers (IE11, various SmartTVs) choke on 204 }; app.use(cors()) // this must be called at the **top** of our file
Now, when we configure our endpoints we can do it like this:
app.post('/api/tasks', cors(corsOptions), (req, res) => { //do something })
As you know from our SSR lab, NextJS will pre-render the page on each request using the data returned by getServerSideProps(). But what if we want to change our data? Let's say we have a list of items that is pre-rendered, if we want to add a new item to it, the changes will only be visible after we reload the page. If we want the changes to be visible right away we can do a nifty trick to solve our problem.
import { useRouter } from 'next/router'; function SomePage(props) { const router = useRouter(); // Call this function whenever you want to // refresh props! const refreshData = () => { router.replace(router.asPath); } } export async function getServerSideProps(context) { // Database logic here }
The refreshData function would be called whenever you want to pull new data from the backend. It'll vary based on your usecase.
But why does it work? Our solution works because we're performing a client-side transition to the same route. router.asPath is a reference to the current path. If we're on /tasks, we're telling Next to do a client-side redirect to /tasks, which causes it to re-fetch the data as JSON, and pass it to the current page as props.
router.replace is like router.push, but it doesn't add an item to the history stack. We don't want this to “count” as a redirect, so that the browser's “Back” button still works as we intend.
You now have a fully functioning backend service that can talk to a database. Using Next, Express and Mongo, create a todo list that fetches, displays and add tasks to the list.
Please take a minute to fill in the feedback form for this lab.